-- Logs begin at Fri 2022-09-23 14:32:25 UTC, end at Fri 2022-09-23 15:43:49 UTC. -- Sep 23 15:03:21.186755 dv-cinder-855013-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 23 15:03:21.194507 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 23 15:03:21 2022] *** Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: nodename: dv-cinder-855013-3-iscsi Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: machine: x86_64 Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: clock source: unix Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: pcre jit disabled Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: detected number of CPU cores: 4 Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: current working directory: / Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: detected binary path: /usr/bin/uwsgi-core Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: your processes number limit is 47680 Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: your memory page size is 4096 bytes Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: detected max file descriptor number: 2048 Sep 23 15:03:21.196539 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: lock engine: pthread robust mutexes Sep 23 15:03:21.199152 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: thunder lock: enabled Sep 23 15:03:21.199152 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 23 15:03:21.199152 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 23 15:03:21.239180 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: Python main interpreter initialized at 0x5609dd0289c0 Sep 23 15:03:21.239180 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: python threads support enabled Sep 23 15:03:21.239180 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: your server socket listen backlog is limited to 100 connections Sep 23 15:03:21.239180 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: your mercy for graceful operations on workers is 90 seconds Sep 23 15:03:21.239575 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: mapped 403077 bytes (393 KB) for 2 cores Sep 23 15:03:21.239655 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: *** Operational MODE: preforking *** Sep 23 15:03:21.239916 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: *** uWSGI is running in multiple interpreter mode *** Sep 23 15:03:21.239916 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: spawned uWSGI master process (pid: 81851) Sep 23 15:03:21.240139 dv-cinder-855013-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 23 15:03:21.241243 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: spawned uWSGI worker 1 (pid: 81864, cores: 1) Sep 23 15:03:21.241243 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: spawned uWSGI worker 2 (pid: 81865, cores: 1) Sep 23 15:03:21.241243 dv-cinder-855013-3-iscsi devstack@keystone.service[81851]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 23 15:03:25.499459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 23 15:03:25.499459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: from cryptography.utils import int_from_bytes Sep 23 15:03:25.500295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 23 15:03:25.500295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: from cryptography.utils import int_from_bytes Sep 23 15:03:25.500822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 23 15:03:25.500822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: from cryptography.utils import int_from_bytes Sep 23 15:03:25.500931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 23 15:03:25.500931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: from cryptography.utils import int_from_bytes Sep 23 15:03:25.549325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 23 15:03:25.549480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 23 15:03:25.549685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 23 15:03:25.549829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] command line args: [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 23 15:03:25.549909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 23 15:03:25.549975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 23 15:03:25.550042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] command line args: [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 23 15:03:25.550099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ================================================================================ {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 23 15:03:25.550306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 23 15:03:25.550439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] admin_token = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.550512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ================================================================================ {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 23 15:03:25.550582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] config_dir = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.550721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.550850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] admin_token = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.550938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] config_source = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.551011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] config_dir = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.551079 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] debug = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.551151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.551227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.551451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] config_source = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.551451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] debug = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.551558 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.551627 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.551709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.551850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.551931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.551996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.552207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.552315 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.552411 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.552486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.552583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_config_append = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.552755 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.552882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.552948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_config_append = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.553018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_dir = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.553081 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.553144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_file = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.553273 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_dir = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.553362 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.553423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_file = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.553541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.553628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.553809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.553924 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.554006 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.554075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.554300 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.554432 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.554517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.554587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.554653 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.554710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.554821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.554964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.555069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.555145 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.555355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.555541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.555649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.555754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.555837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.555909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.555968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.556023 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.556154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.556310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.556415 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.556516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.556711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.556914 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.75/identity {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.557090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] publish_errors = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.557229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.75/identity {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.557305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.557373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] publish_errors = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.557524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.557609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.557685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.557791 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.557920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.557996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.558144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.558233 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.558371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] standard_threads = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.558447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.558595 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.558682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] standard_threads = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.558878 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.558965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.559156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.559156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] use_journal = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.559759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] use_json = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.559900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.559971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] use_stderr = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.560046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.560116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] use_syslog = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.560184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] use_journal = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.560258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.560338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] use_json = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.560338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] use_stderr = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.560462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.560530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] use_syslog = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.560637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.560711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 15:03:25.560792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.560870 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.560940 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.561007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.561081 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.561242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.561327 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.561483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.561560 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.561731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.561812 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.561966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.562054 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.562207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.562283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.562443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.562524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.562669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.562753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.562909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.562992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.563137 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.563219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.563378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.563465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.563617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.563694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.563845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.563921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.564091 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.564187 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.564335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.564418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.564567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.564669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.564824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.564984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.565065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.565140 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.565311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.565395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.565546 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.565623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.565788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.565865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.566011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.566098 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.566235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.566319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.566459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.566536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.566922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.567026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.567098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.567162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.567406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.567468 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.567532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.567589 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.567653 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.567717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.567802 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.567945 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.568041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.568177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.568272 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.568408 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.568536 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.568679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.568758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.568930 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.569020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.569169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.569246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.569399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.569473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.569622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.569706 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.569845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.569946 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.570067 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.570143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.570295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.570449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.570522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.570590 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.570742 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.570818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.570967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.571053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.571190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.571267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.571419 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.571502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.571646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.571730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.571876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.571958 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.572098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.572173 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.572328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.572411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.572555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.572643 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.572807 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.572884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.573035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.573116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.573345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.573431 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.573525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.573595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.573716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.573828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.573924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.573985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.574161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.574244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.574391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.574467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.574737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.574884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.575183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.575320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.575447 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.575559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.575711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.575844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.575948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.576095 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.576190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.576368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.576452 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.576622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.576705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.576871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.576949 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.577116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.577201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.577350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.577464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.577624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.577767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.577901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.578025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.578156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.578278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.578399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.578520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.578653 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.578776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.578908 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.579032 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.579167 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.579289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.579422 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.579547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.579671 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.579797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.579916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.580036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.580173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.580305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.580433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.580562 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.580746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.580884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.581019 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.581144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.581267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.581387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.581512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.581634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.581766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.581891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.582015 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.582138 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.582261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.582386 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.582582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.582739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.582859 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.582983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.583107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.583238 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.583369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.583500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.583623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.583752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.583875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.584034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.584152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.584285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.584409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.584543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.584694 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.584819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.584942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.585080 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.585203 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.585326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.585448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.585572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.585692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.585826 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.585948 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.586071 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.586195 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.586327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.586448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.external = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.586606 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.586742 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.586875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.586998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.587122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.external = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.587247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.587370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.587499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.password = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.587629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.587753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] auth.token = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.587882 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.588016 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.588139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.password = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.588264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.588395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] auth.token = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.588519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.588689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.588810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.588934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.589058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.589196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.589322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.589443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.589566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.589566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.589811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.589947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.590119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.590119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.590860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.591003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.591130 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.591264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.591386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.591386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.591576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.591664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.591728 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.591794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.591981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.592063 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.592124 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.592250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.592374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.592508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.592646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.592767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.592891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.593014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.593139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.593271 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.593397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.593535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.593658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.593796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.593916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.594049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.594172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.594296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.594418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.594543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.594666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.594797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.594797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.595030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.595164 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.595164 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.595396 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.595527 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.595653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.595777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.595777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.596015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.596148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.596271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.596394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.596394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.596654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.596775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.596920 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.597061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.597061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.597306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.597427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.597427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.597739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.597864 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.597987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.598111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.598111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.598344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.598474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.598474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.598745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.598867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.598867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.599102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.599233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.599233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.599467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.599606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.599606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.599850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.599986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.600111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.600246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.600369 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.600565 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.600707 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.600797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.600871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.600942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601083 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601153 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601445 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601585 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.601960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602241 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602733 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602803 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.602952 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.603025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.603095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.603165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.603234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.603303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.603381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.603483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.603587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.603718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.603802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.603959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.604056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.604248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.604330 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.604498 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.604575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.604755 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.604834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.605009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.605087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.605258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.605335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.605495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.605572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.605729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.605851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.606030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.606163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.606262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.606481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.606560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.606735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.606931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.607072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.607229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.607358 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.607486 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.607610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.607699 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.607826 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.607917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.607917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.608145 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.608228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.608291 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.608349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.608423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.608563 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.608741 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.608809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.609328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.609465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.609607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.609734 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.609859 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.609983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.610110 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.610236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.610378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.610503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.610627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.610764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.610889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.611012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.611143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.611269 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.611394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.611519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.611645 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.611771 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.611903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.612029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.612154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.612154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.612408 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.612545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.612700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.612827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.612953 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.613079 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.613210 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.613210 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.613458 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.613458 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.613695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.613827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.613950 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.614075 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.614217 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.614346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.614424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.614496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.614574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.614648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.614727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.614801 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.614876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.614946 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.615017 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.615087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.615165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.615238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.615310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.615380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.615451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.615624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.615765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.615848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.615921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.616087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.616214 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.616307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.616397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.616552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.616702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.616783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.616856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.617037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.617184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.617267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.617340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.617486 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] role.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.617613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.617693 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] role.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.617781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.617937 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.618064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.618143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.618216 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.618406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.618547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.618631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.618704 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.618862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.618996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.619161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.619242 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.619315 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.619387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.619557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.619682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.619777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.619854 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.620006 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.620133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.620212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.620285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.620472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.620678 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.620678 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.620825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.620956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.621080 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.621181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.621405 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.621536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.621820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] role.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.621988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.622161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] role.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.622299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.622422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.622560 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.622702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.622831 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.622959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.623095 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.623224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.623350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.623475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.623601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.623736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.623870 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.623996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.624122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.624122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.624357 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.624481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.624637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.624762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.624888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.625016 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.625140 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.625265 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.625411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.625538 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.625661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.625785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.625907 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.626040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.626040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.626276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.626402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.626525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.626650 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.626781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.626906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.627044 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.627169 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.627293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.627416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.627547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.627672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.627798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.627923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.628047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.628170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.628302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.628426 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.628550 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.628719 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.628861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.628988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.629123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.629248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.629372 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.629495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.629619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.629744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.629875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.629999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.630123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.630249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.630385 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.630526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.630653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.630783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.630908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.631031 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.connection = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.631155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.631279 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.631404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.631534 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.631659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.631783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.631906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.632031 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.632157 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.632298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.632423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.632548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.632687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.632813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.632939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.633074 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.633199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.633326 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.633451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.633576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.633701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.633834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.633960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.634096 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.634223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.634349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.634464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.634597 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.634721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.634844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.634966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.635097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.635097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.635345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.635470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.635600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.635740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81864) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 23 15:03:25.635865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.connection = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.635865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.635865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.635865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.635865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.635865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.635865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.635865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.636776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.636776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.636776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.636776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.637287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.637465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.637587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.637858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.638070 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.638307 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.638545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 15:03:25.638716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81865) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 23 15:03:25.663552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.663775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.663973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.664536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.664777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.665028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.672461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.673720 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.700817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.701008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.701956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.702183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.754904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.755084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.803260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.803466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.804077 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.804313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.960148 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.960356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.960574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.960820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.961002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.961252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.961436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.961691 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.961904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.962152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.962312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.962553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.985131 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.985299 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.985383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.985521 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.985617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.985770 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.985896 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.985976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.986064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.986171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.986258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.986415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.986542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.986637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.997900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:25.998286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.010943 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.011093 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.011215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.011316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.011439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.011516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.011623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.011711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.011891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.012130 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.012130 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81865) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.012868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.012868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.012868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81864) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 15:03:26.028928 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.029109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 23 15:03:26.029477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.029656 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 23 15:03:26.029788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.029890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 23 15:03:26.029956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.030193 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.030319 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.030396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 23 15:03:26.030472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.030612 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.030694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.030898 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.031150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.031451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.040228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.040505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.040806 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.041089 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.041342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.041666 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.041930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.042209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.048213 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.049518 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.051111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.051397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.051681 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.051970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.052256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.052380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.052472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.052773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.052901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.053100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.053227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.053379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.053464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.053563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.053651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.053857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.053996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.054236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.054561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.054837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.055114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.055356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.076840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.077766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.078341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.078624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.079225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.079508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.083346 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.083629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.083914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.084115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.084377 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.084678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.090312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.090606 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.090924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.091226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.096655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.096975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.097180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.097306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.097427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.097572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.097684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.097800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.097916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.098031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.098159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.098278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.098394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.098588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.098740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.098903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.114303 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.114591 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.114716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.114913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.118441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.118675 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.121725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.122023 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.122164 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.122423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.126310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.126570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.129478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.129725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.132712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.132901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.132997 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.133216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.133341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.133478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.133572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.133760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.133885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.134022 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.134102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.134320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.149873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.150001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.150138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.150261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.150404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.150542 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.150623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.150771 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.157457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.157758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.158993 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.159285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.160498 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.160789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.160912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.161116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.163557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.163812 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.163938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.164093 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.164221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.164435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.164559 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.164656 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.164879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.165083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.176438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.176745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.176971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.177096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.177232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.177333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.177518 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.177644 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.177779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.177984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.178120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.178254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.178431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.178708 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.193470 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.193821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.196349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.196694 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.199222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.199677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.200996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.201153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.202475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.202603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.202745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.202829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.203181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.203309 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.209491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.209618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.210918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.211065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.213774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.213902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.214162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.214289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.214446 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.214854 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.215030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.215201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.222566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.222695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.222900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.223030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.223165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.223245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.229878 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81864) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.230037 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81865) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 15:03:26.230123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.230252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.230375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.230568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.230750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.230923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.231089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.231214 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.231389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.231492 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.231673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.231776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.231975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.232100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.232253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.232336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.232492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.232653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.232776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.232886 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.233041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.233174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.233361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.233486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.233641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81864) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.233780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81865) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 15:03:26.267917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5609dd0289c0 pid: 81864 (default app) Sep 23 15:03:26.268073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5609dd0289c0 pid: 81865 (default app) Sep 23 15:03:26.271260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaff5b1-a535-44ea-b95f-8fb8a628c413 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:26.271541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaff5b1-a535-44ea-b95f-8fb8a628c413 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:26.271803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaff5b1-a535-44ea-b95f-8fb8a628c413 None None] PATH_INFO: `/v3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:26.275955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1/1] 10.222.0.75 () {50 vars in 791 bytes} [Fri Sep 23 15:03:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:33.092991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39bc6f-2418-47a1-a613-a2b64b17c5ee None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:33.093552 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39bc6f-2418-47a1-a613-a2b64b17c5ee None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:33.093552 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39bc6f-2418-47a1-a613-a2b64b17c5ee None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:33.097768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1/2] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:03:33.103856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121233f7-bd81-45c0-aab1-d24637a3629e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:33.104329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121233f7-bd81-45c0-aab1-d24637a3629e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:33.104468 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121233f7-bd81-45c0-aab1-d24637a3629e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:33.176574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: INFO dbcounter [None req-121233f7-bd81-45c0-aab1-d24637a3629e None None] Registered counter for database keystone Sep 23 15:03:33.196841 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG oslo_db.sqlalchemy.engines [None req-121233f7-bd81-45c0-aab1-d24637a3629e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81864) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 23 15:03:33.204639 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writer thread running {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 23 15:03:33.387678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG passlib.handlers.bcrypt [None req-121233f7-bd81-45c0-aab1-d24637a3629e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81864) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 23 15:03:33.387957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG passlib.handlers.bcrypt [None req-121233f7-bd81-45c0-aab1-d24637a3629e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81864) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 23 15:03:33.446639 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-121233f7-bd81-45c0-aab1-d24637a3629e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:33.448270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-121233f7-bd81-45c0-aab1-d24637a3629e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:33.584331 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 2/3] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 23 15:03:34.030390 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39bc6f-2418-47a1-a613-a2b64b17c5ee None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:34.100327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: INFO dbcounter [None req-de39bc6f-2418-47a1-a613-a2b64b17c5ee None None] Registered counter for database keystone Sep 23 15:03:34.122872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG oslo_db.sqlalchemy.engines [None req-de39bc6f-2418-47a1-a613-a2b64b17c5ee None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81865) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 23 15:03:34.130360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writer thread running {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 23 15:03:34.248622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39bc6f-2418-47a1-a613-a2b64b17c5ee None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:34.253379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb0e71-fb19-4e88-ac16-d086b60955fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:34.254006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb0e71-fb19-4e88-ac16-d086b60955fe admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:34.254278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb0e71-fb19-4e88-ac16-d086b60955fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:34.254543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb0e71-fb19-4e88-ac16-d086b60955fe admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:34.263417 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb0e71-fb19-4e88-ac16-d086b60955fe admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:34.314459 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: WARNING keystone.common.rbac_enforcer.enforcer [None req-88bb0e71-fb19-4e88-ac16-d086b60955fe admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 23 15:03:34.314800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb0e71-fb19-4e88-ac16-d086b60955fe admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:34.325341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 2/4] 10.222.0.75 () {58 vars in 1324 bytes} [Fri Sep 23 15:03:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:03:34.331587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121233f7-bd81-45c0-aab1-d24637a3629e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:34.349778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121233f7-bd81-45c0-aab1-d24637a3629e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:34.355476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09baed25-0b97-46b5-9b45-0ab32d403122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:34.357174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baed25-0b97-46b5-9b45-0ab32d403122 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:34.357454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baed25-0b97-46b5-9b45-0ab32d403122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:34.357732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09baed25-0b97-46b5-9b45-0ab32d403122 admin admin] PATH_INFO: `/v3/projects/a6e3ae74e78241ac8bfc0ae2c7103bec` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:34.359731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09baed25-0b97-46b5-9b45-0ab32d403122 admin admin] RBAC: Authorizing `identity:get_project(project_id=a6e3ae74e78241ac8bfc0ae2c7103bec)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:34.413390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: WARNING keystone.common.rbac_enforcer.enforcer [None req-09baed25-0b97-46b5-9b45-0ab32d403122 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 23 15:03:34.413727 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09baed25-0b97-46b5-9b45-0ab32d403122 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:34.418984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 3/5] 10.222.0.75 () {56 vars in 1207 bytes} [Fri Sep 23 15:03:34 2022] GET /identity/v3/projects/a6e3ae74e78241ac8bfc0ae2c7103bec => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:36.545530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3b38d-652a-4c93-866d-3b272b8c68c6 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:36.546077 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3b38d-652a-4c93-866d-3b272b8c68c6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:36.546077 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3b38d-652a-4c93-866d-3b272b8c68c6 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:36.550161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 3/6] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:36.555936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083f2b8-d2ef-4413-9d8e-d6e19a51acc5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:36.556376 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083f2b8-d2ef-4413-9d8e-d6e19a51acc5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:36.556458 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2083f2b8-d2ef-4413-9d8e-d6e19a51acc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:36.577340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2083f2b8-d2ef-4413-9d8e-d6e19a51acc5 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:36.578866 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2083f2b8-d2ef-4413-9d8e-d6e19a51acc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:36.587263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 4/7] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 23 15:03:36.848353 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3b38d-652a-4c93-866d-3b272b8c68c6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:36.862113 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3b38d-652a-4c93-866d-3b272b8c68c6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:36.866595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc30ac-721e-4844-bb90-e88cde8a3f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:36.867230 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc30ac-721e-4844-bb90-e88cde8a3f39 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:36.867499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc30ac-721e-4844-bb90-e88cde8a3f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:36.867763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc30ac-721e-4844-bb90-e88cde8a3f39 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:36.875844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc30ac-721e-4844-bb90-e88cde8a3f39 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:36.877648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc30ac-721e-4844-bb90-e88cde8a3f39 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:36.886255 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 4/8] 10.222.0.75 () {58 vars in 1324 bytes} [Fri Sep 23 15:03:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:03:36.892525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083f2b8-d2ef-4413-9d8e-d6e19a51acc5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:36.902462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2083f2b8-d2ef-4413-9d8e-d6e19a51acc5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:36.908399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947150c-aa73-472c-afaf-d5153abb497d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:36.909068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947150c-aa73-472c-afaf-d5153abb497d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:36.909347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947150c-aa73-472c-afaf-d5153abb497d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:36.909611 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947150c-aa73-472c-afaf-d5153abb497d admin admin] PATH_INFO: `/v3/users/1d564d82faf2482faf5205415837bf68` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:36.911104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947150c-aa73-472c-afaf-d5153abb497d admin admin] RBAC: Authorizing `identity:get_user(user_id=1d564d82faf2482faf5205415837bf68)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:36.913027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947150c-aa73-472c-afaf-d5153abb497d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:36.914611 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 5/9] 10.222.0.75 () {56 vars in 1198 bytes} [Fri Sep 23 15:03:36 2022] GET /identity/v3/users/1d564d82faf2482faf5205415837bf68 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:39.129640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3c4e1-5423-4c95-92ce-d121bda1d84d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:39.130259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3c4e1-5423-4c95-92ce-d121bda1d84d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:39.130259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3c4e1-5423-4c95-92ce-d121bda1d84d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:39.131455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 5/10] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:39.137575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4edf1a-43b3-4441-b8f4-899f0a9570e8 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:39.138023 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4edf1a-43b3-4441-b8f4-899f0a9570e8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:39.138100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4edf1a-43b3-4441-b8f4-899f0a9570e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:39.158158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1f4edf1a-43b3-4441-b8f4-899f0a9570e8 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:39.159342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1f4edf1a-43b3-4441-b8f4-899f0a9570e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:39.170799 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a0ec7-7073-4cc3-934c-f392646aae3d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:39.171073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a0ec7-7073-4cc3-934c-f392646aae3d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:39.171338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a0ec7-7073-4cc3-934c-f392646aae3d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:39.172693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 6/11] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:39.177871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7064fb-38f4-4346-8a05-750cafc04348 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:39.178152 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7064fb-38f4-4346-8a05-750cafc04348 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:39.178415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7064fb-38f4-4346-8a05-750cafc04348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:39.222078 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG passlib.handlers.bcrypt [None req-7f7064fb-38f4-4346-8a05-750cafc04348 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81865) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 23 15:03:39.222468 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG passlib.handlers.bcrypt [None req-7f7064fb-38f4-4346-8a05-750cafc04348 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81865) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 23 15:03:39.247355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 6/12] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:03:39.272172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7f7064fb-38f4-4346-8a05-750cafc04348 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:39.273802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7f7064fb-38f4-4346-8a05-750cafc04348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:39.380334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 7/13] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:03:39.515451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4edf1a-43b3-4441-b8f4-899f0a9570e8 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:39.543116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:39.629494 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4edf1a-43b3-4441-b8f4-899f0a9570e8 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:39.647916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7064fb-38f4-4346-8a05-750cafc04348 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:39.666717 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54363-7f8e-42ac-aa97-17dec03362bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:39.667390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54363-7f8e-42ac-aa97-17dec03362bc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:39.668830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54363-7f8e-42ac-aa97-17dec03362bc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:39.668830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54363-7f8e-42ac-aa97-17dec03362bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:39.679920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d54363-7f8e-42ac-aa97-17dec03362bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-f4d54363-7f8e-42ac-aa97-17dec03362bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:03:39.688867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:39.689840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:39.690665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:39.690665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:39.690665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:39.690665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:39.690665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:03:39.690665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:03:39.690665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:03:39.690665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:03:39.690665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:03:39.690665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:03:39.690665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:39.690665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:03:39.690665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 7/14] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:03:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:39.695781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54363-7f8e-42ac-aa97-17dec03362bc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:39.756013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7064fb-38f4-4346-8a05-750cafc04348 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:39.758296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:39.778266 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:39.798456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b57b1-5069-498d-99d8-d8518637bec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:39.799223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54363-7f8e-42ac-aa97-17dec03362bc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:39.799324 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b57b1-5069-498d-99d8-d8518637bec4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:39.799612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b57b1-5069-498d-99d8-d8518637bec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:39.799879 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b57b1-5069-498d-99d8-d8518637bec4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:39.836005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca122b38-44e6-4012-8cff-c7d4605ed808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:39.836631 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca122b38-44e6-4012-8cff-c7d4605ed808 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:39.836908 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca122b38-44e6-4012-8cff-c7d4605ed808 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:39.837174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca122b38-44e6-4012-8cff-c7d4605ed808 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:39.837855 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca122b38-44e6-4012-8cff-c7d4605ed808 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:39.838154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca122b38-44e6-4012-8cff-c7d4605ed808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:39.839643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca122b38-44e6-4012-8cff-c7d4605ed808 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:39.852253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 8/15] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:03:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:39.858336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca122b38-44e6-4012-8cff-c7d4605ed808 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:39.869663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b57b1-5069-498d-99d8-d8518637bec4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:39.871310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b57b1-5069-498d-99d8-d8518637bec4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:39.965738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca122b38-44e6-4012-8cff-c7d4605ed808 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:39.975783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 8/16] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:03:39.981722 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b57b1-5069-498d-99d8-d8518637bec4 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:39.984880 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:40.001471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b69c20-2032-426f-829a-452cc5c149d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:40.002113 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b69c20-2032-426f-829a-452cc5c149d7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:40.002385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b69c20-2032-426f-829a-452cc5c149d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:40.002650 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b69c20-2032-426f-829a-452cc5c149d7 None admin] PATH_INFO: `/v3/users/1d564d82faf2482faf5205415837bf68` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:40.003955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b69c20-2032-426f-829a-452cc5c149d7 None admin] RBAC: Authorizing `identity:get_user(user_id=1d564d82faf2482faf5205415837bf68)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:40.005457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b69c20-2032-426f-829a-452cc5c149d7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:40.007053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 9/17] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:03:39 2022] GET /identity/v3/users/1d564d82faf2482faf5205415837bf68 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:40.012835 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b69c20-2032-426f-829a-452cc5c149d7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:40.028946 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:40.081756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b57b1-5069-498d-99d8-d8518637bec4 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:40.113841 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b69c20-2032-426f-829a-452cc5c149d7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:40.118042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243e600-000b-4fec-8a3f-1eea1b2facee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:40.118958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e600-000b-4fec-8a3f-1eea1b2facee None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:40.119242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e600-000b-4fec-8a3f-1eea1b2facee None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:40.119510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243e600-000b-4fec-8a3f-1eea1b2facee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:40.138717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243e600-000b-4fec-8a3f-1eea1b2facee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:40.140187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243e600-000b-4fec-8a3f-1eea1b2facee None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:40.148806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95271fa3-0ee7-4a98-a9cd-7a15da03ba49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:40.149444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95271fa3-0ee7-4a98-a9cd-7a15da03ba49 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:40.149734 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95271fa3-0ee7-4a98-a9cd-7a15da03ba49 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:40.149982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95271fa3-0ee7-4a98-a9cd-7a15da03ba49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:40.151105 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95271fa3-0ee7-4a98-a9cd-7a15da03ba49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:40.152881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95271fa3-0ee7-4a98-a9cd-7a15da03ba49 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:40.154433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 10/18] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-8243e600-000b-4fec-8a3f-1eea1b2facee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:03:40.159268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:40.160163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:03:40.160983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:03:40.160983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:40.160983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:03:40.161147 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95271fa3-0ee7-4a98-a9cd-7a15da03ba49 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:40.161217 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 9/19] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:40.166085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243e600-000b-4fec-8a3f-1eea1b2facee None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:40.224109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:40.243547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:40.264155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95271fa3-0ee7-4a98-a9cd-7a15da03ba49 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:40.270272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243e600-000b-4fec-8a3f-1eea1b2facee None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:40.299258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed0167-39e7-4b14-bf53-7fb54b2eb6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:40.299865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed0167-39e7-4b14-bf53-7fb54b2eb6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:40.300152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed0167-39e7-4b14-bf53-7fb54b2eb6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:40.300422 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed0167-39e7-4b14-bf53-7fb54b2eb6b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:40.301196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed0167-39e7-4b14-bf53-7fb54b2eb6b4 None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.domain.id=default, user.id=1d564d82faf2482faf5205415837bf68) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:40.301491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed0167-39e7-4b14-bf53-7fb54b2eb6b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:40.302973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed0167-39e7-4b14-bf53-7fb54b2eb6b4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:40.307319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7f898-f1a4-4dc2-8972-887766dbcd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:40.307929 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7f898-f1a4-4dc2-8972-887766dbcd49 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:40.308273 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7f898-f1a4-4dc2-8972-887766dbcd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:40.308981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7f898-f1a4-4dc2-8972-887766dbcd49 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:40.309657 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f7f898-f1a4-4dc2-8972-887766dbcd49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:40.309956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f7f898-f1a4-4dc2-8972-887766dbcd49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:40.310157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 11/20] 10.222.0.75 () {56 vars in 1322 bytes} [Fri Sep 23 15:03:40 2022] GET /identity/v3/role_assignments?user.id=1d564d82faf2482faf5205415837bf68&scope.domain.id=default&role.id=ae444a489623434999d758b71ae2dc87 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:40.311665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f7f898-f1a4-4dc2-8972-887766dbcd49 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:40.333450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 10/21] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:03:42.655171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21d2ba-bc84-43f0-9cab-6750a06b80c5 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:42.656209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21d2ba-bc84-43f0-9cab-6750a06b80c5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:42.656715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21d2ba-bc84-43f0-9cab-6750a06b80c5 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:42.658283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 12/22] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:42.663904 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc80a6-e169-4050-ae37-8d04a76148de None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:42.664792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc80a6-e169-4050-ae37-8d04a76148de None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:42.665287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc80a6-e169-4050-ae37-8d04a76148de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:42.684561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-54bc80a6-e169-4050-ae37-8d04a76148de None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:42.686126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-54bc80a6-e169-4050-ae37-8d04a76148de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:42.774865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 11/23] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:03:43.074467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4d4ec-e182-46ee-a7be-0001caf2b96e None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:43.075146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4d4ec-e182-46ee-a7be-0001caf2b96e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:43.075630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4d4ec-e182-46ee-a7be-0001caf2b96e None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:43.077201 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 13/24] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:43.083163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3530d7a-5ae0-49e0-90b9-869f93f77825 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:43.083797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3530d7a-5ae0-49e0-90b9-869f93f77825 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:43.084266 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3530d7a-5ae0-49e0-90b9-869f93f77825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:43.089984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4d4ec-e182-46ee-a7be-0001caf2b96e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:43.113573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d3530d7a-5ae0-49e0-90b9-869f93f77825 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:43.116298 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d3530d7a-5ae0-49e0-90b9-869f93f77825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:43.197695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:43.253027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 12/25] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:03:43.254123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c96485-0d37-48ba-954c-1b6d5cb51605 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:43.254406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c96485-0d37-48ba-954c-1b6d5cb51605 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:43.254683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c96485-0d37-48ba-954c-1b6d5cb51605 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:43.255899 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 13/26] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:43.262941 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1e108-5e86-4ff3-a65a-d120261c5dc9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:43.263398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1e108-5e86-4ff3-a65a-d120261c5dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:43.264481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1e108-5e86-4ff3-a65a-d120261c5dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:43.293256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0ce1e108-5e86-4ff3-a65a-d120261c5dc9 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:43.294686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0ce1e108-5e86-4ff3-a65a-d120261c5dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:43.322220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4d4ec-e182-46ee-a7be-0001caf2b96e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:43.367590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 14/27] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:03:43.398672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b0db7-8ba9-4948-aac7-6436a3868b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:43.399317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b0db7-8ba9-4948-aac7-6436a3868b5d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:43.399590 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b0db7-8ba9-4948-aac7-6436a3868b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:43.399854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b0db7-8ba9-4948-aac7-6436a3868b5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:43.418061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b0db7-8ba9-4948-aac7-6436a3868b5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-9f7b0db7-8ba9-4948-aac7-6436a3868b5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:03:43.424619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:43.425466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:43.426209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:43.426209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:43.426209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:43.426209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:43.426209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:03:43.426209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:03:43.426209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:03:43.426209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:03:43.426209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:03:43.426209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:03:43.426209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:43.426209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:03:43.426209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 14/28] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:03:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:03:43.432866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1e108-5e86-4ff3-a65a-d120261c5dc9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:43.593641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1e108-5e86-4ff3-a65a-d120261c5dc9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:43.609966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:43.645607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5721d7a-385c-4f46-be30-dfded4561be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:43.646136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5721d7a-385c-4f46-be30-dfded4561be6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:43.646411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5721d7a-385c-4f46-be30-dfded4561be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:43.646682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5721d7a-385c-4f46-be30-dfded4561be6 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:43.647363 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5721d7a-385c-4f46-be30-dfded4561be6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:43.647663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5721d7a-385c-4f46-be30-dfded4561be6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:43.649140 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5721d7a-385c-4f46-be30-dfded4561be6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:43.667649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 15/29] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:03:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:43.672044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b0db7-8ba9-4948-aac7-6436a3868b5d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:43.679545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:43.807348 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5721d7a-385c-4f46-be30-dfded4561be6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:43.837032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b0db7-8ba9-4948-aac7-6436a3868b5d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:43.902716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6d09f-22f8-4c9d-aee2-30e4e95122d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:43.903991 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6d09f-22f8-4c9d-aee2-30e4e95122d6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:43.904643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6d09f-22f8-4c9d-aee2-30e4e95122d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:43.905126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6d09f-22f8-4c9d-aee2-30e4e95122d6 None admin] PATH_INFO: `/v3/users/1d564d82faf2482faf5205415837bf68` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:43.907045 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6d09f-22f8-4c9d-aee2-30e4e95122d6 None admin] RBAC: Authorizing `identity:get_user(user_id=1d564d82faf2482faf5205415837bf68)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:43.909541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6d09f-22f8-4c9d-aee2-30e4e95122d6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:43.911779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 15/30] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:03:43 2022] GET /identity/v3/users/1d564d82faf2482faf5205415837bf68 => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:43.913451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614ab7f-452a-4f2f-b0f2-a907766702c5 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:43.913909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614ab7f-452a-4f2f-b0f2-a907766702c5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:43.914369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614ab7f-452a-4f2f-b0f2-a907766702c5 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:43.916467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 16/31] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:43.917664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614ab7f-452a-4f2f-b0f2-a907766702c5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:43.951735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5721d7a-385c-4f46-be30-dfded4561be6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:43.996111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9a8f8-8d20-4851-b9d9-c8d97edd7f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:43.996855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9a8f8-8d20-4851-b9d9-c8d97edd7f0a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:43.997288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9a8f8-8d20-4851-b9d9-c8d97edd7f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:43.997691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9a8f8-8d20-4851-b9d9-c8d97edd7f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:44.009126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9a8f8-8d20-4851-b9d9-c8d97edd7f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:44.010872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9a8f8-8d20-4851-b9d9-c8d97edd7f0a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:44.013482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-d9d9a8f8-8d20-4851-b9d9-c8d97edd7f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:03:44.029418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:44.030313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:03:44.031109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:03:44.031109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:03:44.031109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:03:44.033129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 16/32] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:03:44.034126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9a8f8-8d20-4851-b9d9-c8d97edd7f0a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:44.060640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614ab7f-452a-4f2f-b0f2-a907766702c5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:44.065829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:44.111248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448f0ac-11ee-4129-b948-ede09083ad53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:44.112248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448f0ac-11ee-4129-b948-ede09083ad53 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:44.112619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448f0ac-11ee-4129-b948-ede09083ad53 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:44.112965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448f0ac-11ee-4129-b948-ede09083ad53 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:44.113736 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448f0ac-11ee-4129-b948-ede09083ad53 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:44.115450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448f0ac-11ee-4129-b948-ede09083ad53 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:44.156018 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9a8f8-8d20-4851-b9d9-c8d97edd7f0a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:44.198299 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be08b70e-fb24-4c27-b46c-94d9960abaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:44.199274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08b70e-fb24-4c27-b46c-94d9960abaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:44.199746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08b70e-fb24-4c27-b46c-94d9960abaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:44.200193 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be08b70e-fb24-4c27-b46c-94d9960abaf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:44.201789 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08b70e-fb24-4c27-b46c-94d9960abaf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:44.203335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be08b70e-fb24-4c27-b46c-94d9960abaf0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:44.205133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 17/33] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:44.206321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121d552-1a02-4f9c-a240-2b1097237d14 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:44.206715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121d552-1a02-4f9c-a240-2b1097237d14 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:44.207058 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121d552-1a02-4f9c-a240-2b1097237d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:44.228799 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d121d552-1a02-4f9c-a240-2b1097237d14 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:44.230187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d121d552-1a02-4f9c-a240-2b1097237d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:44.256425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 17/34] 10.222.0.75 () {60 vars in 1131 bytes} [Fri Sep 23 15:03:43 2022] POST /identity/v3/roles => generated 219 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:03:44.257785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448f0ac-11ee-4129-b948-ede09083ad53 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:44.310835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 18/35] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:03:44.311775 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121d552-1a02-4f9c-a240-2b1097237d14 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:44.334076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:44.374949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448f0ac-11ee-4129-b948-ede09083ad53 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:44.407143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:44.426310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d4428-8892-472a-9013-7553f757807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:44.427420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d4428-8892-472a-9013-7553f757807f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:44.427930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d4428-8892-472a-9013-7553f757807f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:44.428417 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d4428-8892-472a-9013-7553f757807f None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:44.429338 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d4428-8892-472a-9013-7553f757807f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:44.429911 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d4428-8892-472a-9013-7553f757807f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:44.431786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d4428-8892-472a-9013-7553f757807f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:44.437303 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121d552-1a02-4f9c-a240-2b1097237d14 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:44.470633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 18/36] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:44.478634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d4428-8892-472a-9013-7553f757807f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:44.514461 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b0697-7290-4a72-95d4-eaf154d34604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:44.515663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b0697-7290-4a72-95d4-eaf154d34604 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:44.516204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b0697-7290-4a72-95d4-eaf154d34604 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:44.521628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b0697-7290-4a72-95d4-eaf154d34604 None admin] PATH_INFO: `/v3/domains/default/users/1d564d82faf2482faf5205415837bf68/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:44.529719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b0697-7290-4a72-95d4-eaf154d34604 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1d564d82faf2482faf5205415837bf68, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:44.531629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b0697-7290-4a72-95d4-eaf154d34604 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:44.734580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d4428-8892-472a-9013-7553f757807f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:44.738557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 19/37] 10.222.0.75 () {58 vars in 1361 bytes} [Fri Sep 23 15:03:44 2022] PUT /identity/v3/domains/default/users/1d564d82faf2482faf5205415837bf68/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:03:44.739083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b0697-7290-4a72-95d4-eaf154d34604 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:44.792531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de9672-e83c-426f-b04e-02ebd31342d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:44.793803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de9672-e83c-426f-b04e-02ebd31342d4 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:44.793803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de9672-e83c-426f-b04e-02ebd31342d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:44.794069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de9672-e83c-426f-b04e-02ebd31342d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:44.794820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de9672-e83c-426f-b04e-02ebd31342d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:44.796438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de9672-e83c-426f-b04e-02ebd31342d4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:44.842394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 19/38] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:03:44 2022] POST /identity/v3/projects => generated 301 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:03:44.843949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437d329-1f3b-4593-9aec-555a7d941579 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:44.844235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437d329-1f3b-4593-9aec-555a7d941579 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:44.844504 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437d329-1f3b-4593-9aec-555a7d941579 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:44.845951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 20/39] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:44.847300 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807ad5a-767f-4a4c-8821-52b7ee77ddd6 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:44.847582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807ad5a-767f-4a4c-8821-52b7ee77ddd6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:44.847857 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807ad5a-767f-4a4c-8821-52b7ee77ddd6 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:44.852786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 21/40] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:44.852786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4589c-21ea-4d9e-98e7-2a338505c993 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:44.852786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4589c-21ea-4d9e-98e7-2a338505c993 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:44.852786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4589c-21ea-4d9e-98e7-2a338505c993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:44.863535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:44.884428 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-67e4589c-21ea-4d9e-98e7-2a338505c993 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:44.886447 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-67e4589c-21ea-4d9e-98e7-2a338505c993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:44.910531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b0697-7290-4a72-95d4-eaf154d34604 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:44.963971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:44.967894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b097cd-2b31-4089-9c5d-1a76f79119fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:44.968874 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b097cd-2b31-4089-9c5d-1a76f79119fa None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:44.969527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b097cd-2b31-4089-9c5d-1a76f79119fa None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:44.969948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b097cd-2b31-4089-9c5d-1a76f79119fa None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:44.970767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b097cd-2b31-4089-9c5d-1a76f79119fa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:44.973900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b097cd-2b31-4089-9c5d-1a76f79119fa None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:45.006844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 20/41] 10.222.0.75 () {60 vars in 1131 bytes} [Fri Sep 23 15:03:44 2022] POST /identity/v3/roles => generated 225 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:03:45.009094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899d71e-d7c3-41b6-85d9-b98e0e9e2c85 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:45.009451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899d71e-d7c3-41b6-85d9-b98e0e9e2c85 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:45.010033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899d71e-d7c3-41b6-85d9-b98e0e9e2c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:45.020654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 22/42] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:03:45.036441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0899d71e-d7c3-41b6-85d9-b98e0e9e2c85 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:45.038503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0899d71e-d7c3-41b6-85d9-b98e0e9e2c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:45.131187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 21/43] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:03:45.378930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4589c-21ea-4d9e-98e7-2a338505c993 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:45.404870 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899d71e-d7c3-41b6-85d9-b98e0e9e2c85 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:45.477296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:45.487829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4589c-21ea-4d9e-98e7-2a338505c993 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:45.519293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899d71e-d7c3-41b6-85d9-b98e0e9e2c85 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:45.523384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c906969-381d-44bf-9264-f816be3d6e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:45.524038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c906969-381d-44bf-9264-f816be3d6e68 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:45.524310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c906969-381d-44bf-9264-f816be3d6e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:45.524578 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c906969-381d-44bf-9264-f816be3d6e68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:45.525723 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c906969-381d-44bf-9264-f816be3d6e68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:45.527496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c906969-381d-44bf-9264-f816be3d6e68 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:45.529122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 23/44] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:45.534729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c906969-381d-44bf-9264-f816be3d6e68 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:45.566609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44effdf8-2c46-4909-a8d9-c58808b96145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:45.567320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44effdf8-2c46-4909-a8d9-c58808b96145 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:45.567593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44effdf8-2c46-4909-a8d9-c58808b96145 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:45.567851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44effdf8-2c46-4909-a8d9-c58808b96145 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:45.568522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44effdf8-2c46-4909-a8d9-c58808b96145 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:45.570140 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44effdf8-2c46-4909-a8d9-c58808b96145 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:45.585117 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:45.588332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 22/45] 10.222.0.75 () {60 vars in 1131 bytes} [Fri Sep 23 15:03:45 2022] POST /identity/v3/roles => generated 223 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:03:45.664736 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c906969-381d-44bf-9264-f816be3d6e68 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:45.822061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df486928-d582-4363-8d94-70d25d6c4f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:45.822693 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df486928-d582-4363-8d94-70d25d6c4f2e None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:45.822985 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df486928-d582-4363-8d94-70d25d6c4f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:45.823233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df486928-d582-4363-8d94-70d25d6c4f2e None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:45.824007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df486928-d582-4363-8d94-70d25d6c4f2e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:45.825626 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df486928-d582-4363-8d94-70d25d6c4f2e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:45.854559 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 24/46] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:03:45 2022] POST /identity/v3/projects => generated 312 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:03:47.069382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31be6ca-c4e6-483b-b207-b6ca39d76f68 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:47.070020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31be6ca-c4e6-483b-b207-b6ca39d76f68 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:47.070020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31be6ca-c4e6-483b-b207-b6ca39d76f68 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:47.071518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 23/47] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:47.076860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cc30b-0207-4b68-9742-2d2866380d05 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:47.077283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cc30b-0207-4b68-9742-2d2866380d05 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:47.077405 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cc30b-0207-4b68-9742-2d2866380d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:47.096825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-133cc30b-0207-4b68-9742-2d2866380d05 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:47.098079 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-133cc30b-0207-4b68-9742-2d2866380d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:47.170582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 25/48] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:03:47.436176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31be6ca-c4e6-483b-b207-b6ca39d76f68 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:47.547979 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31be6ca-c4e6-483b-b207-b6ca39d76f68 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:47.575631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:47.585719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60daebf1-3b2e-4d9f-a39e-0358b757033c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:47.586360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daebf1-3b2e-4d9f-a39e-0358b757033c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:47.586639 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daebf1-3b2e-4d9f-a39e-0358b757033c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:47.586912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60daebf1-3b2e-4d9f-a39e-0358b757033c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:47.599876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60daebf1-3b2e-4d9f-a39e-0358b757033c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-60daebf1-3b2e-4d9f-a39e-0358b757033c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:03:47.606006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:47.606859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:47.607766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:47.607766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:47.607766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:47.607766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:47.607766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:03:47.607766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:03:47.607766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:03:47.607766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:03:47.607766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:03:47.607766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:03:47.607766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:47.607766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:03:47.607766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 24/49] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:03:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:47.613656 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cc30b-0207-4b68-9742-2d2866380d05 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:47.640818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:47.720059 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133cc30b-0207-4b68-9742-2d2866380d05 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:47.756143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b9189-e1c1-43a6-8fc8-dabe73246db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:47.756789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b9189-e1c1-43a6-8fc8-dabe73246db1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:47.757068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b9189-e1c1-43a6-8fc8-dabe73246db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:47.757342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b9189-e1c1-43a6-8fc8-dabe73246db1 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:47.758025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b9189-e1c1-43a6-8fc8-dabe73246db1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:47.758328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b9189-e1c1-43a6-8fc8-dabe73246db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:47.759724 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b9189-e1c1-43a6-8fc8-dabe73246db1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:47.768739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 26/50] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:03:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:47.774561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60daebf1-3b2e-4d9f-a39e-0358b757033c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:47.878081 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60daebf1-3b2e-4d9f-a39e-0358b757033c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:47.911292 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57add9-e622-4fdc-812b-90bcb54c1a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:47.911937 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57add9-e622-4fdc-812b-90bcb54c1a1b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:47.912211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57add9-e622-4fdc-812b-90bcb54c1a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:47.912479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57add9-e622-4fdc-812b-90bcb54c1a1b None admin] PATH_INFO: `/v3/users/1d564d82faf2482faf5205415837bf68` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:47.913649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57add9-e622-4fdc-812b-90bcb54c1a1b None admin] RBAC: Authorizing `identity:get_user(user_id=1d564d82faf2482faf5205415837bf68)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:47.915110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57add9-e622-4fdc-812b-90bcb54c1a1b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:47.921511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 25/51] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:03:47 2022] GET /identity/v3/users/1d564d82faf2482faf5205415837bf68 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:47.922759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b9189-e1c1-43a6-8fc8-dabe73246db1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:48.015453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:48.030397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b9189-e1c1-43a6-8fc8-dabe73246db1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:48.067277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88da8e-2786-497f-8bfb-a0b02b7f1aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:48.067907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88da8e-2786-497f-8bfb-a0b02b7f1aed None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:48.068179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88da8e-2786-497f-8bfb-a0b02b7f1aed None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:48.068441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88da8e-2786-497f-8bfb-a0b02b7f1aed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:48.069590 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88da8e-2786-497f-8bfb-a0b02b7f1aed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:48.071028 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88da8e-2786-497f-8bfb-a0b02b7f1aed None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:48.072599 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 27/52] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:48.078594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57add9-e622-4fdc-812b-90bcb54c1a1b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:48.088205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058b56e-c0af-43d2-8be7-1b7754f62228 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:48.088707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058b56e-c0af-43d2-8be7-1b7754f62228 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:48.088780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058b56e-c0af-43d2-8be7-1b7754f62228 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:48.090194 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 28/53] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:03:48.096348 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876f524-1ada-480f-8f7f-1ab87fc0214e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:48.096669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876f524-1ada-480f-8f7f-1ab87fc0214e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:48.096949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876f524-1ada-480f-8f7f-1ab87fc0214e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:48.118172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5876f524-1ada-480f-8f7f-1ab87fc0214e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:48.119448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5876f524-1ada-480f-8f7f-1ab87fc0214e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:48.197160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 29/54] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:03:48.243456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:48.287727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57add9-e622-4fdc-812b-90bcb54c1a1b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:48.321011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17142-52c0-4157-90f7-d3f0c1970bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:48.321617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17142-52c0-4157-90f7-d3f0c1970bfe None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:48.321891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17142-52c0-4157-90f7-d3f0c1970bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:48.322162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17142-52c0-4157-90f7-d3f0c1970bfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:48.322887 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e17142-52c0-4157-90f7-d3f0c1970bfe None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.domain.id=default, user.id=1d564d82faf2482faf5205415837bf68) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:48.323191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e17142-52c0-4157-90f7-d3f0c1970bfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:48.324764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e17142-52c0-4157-90f7-d3f0c1970bfe None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:48.332652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 26/55] 10.222.0.75 () {56 vars in 1322 bytes} [Fri Sep 23 15:03:48 2022] GET /identity/v3/role_assignments?user.id=1d564d82faf2482faf5205415837bf68&scope.domain.id=default&role.id=ae444a489623434999d758b71ae2dc87 => generated 530 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:48.467016 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876f524-1ada-480f-8f7f-1ab87fc0214e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:48.563622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:48.580577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876f524-1ada-480f-8f7f-1ab87fc0214e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:48.622729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b29ec-0e71-4cad-b926-4cca93e88e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:48.623421 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b29ec-0e71-4cad-b926-4cca93e88e49 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:48.623663 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b29ec-0e71-4cad-b926-4cca93e88e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:48.623932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b29ec-0e71-4cad-b926-4cca93e88e49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:48.625153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b29ec-0e71-4cad-b926-4cca93e88e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:48.626624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b29ec-0e71-4cad-b926-4cca93e88e49 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:48.628217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 30/56] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:48.634794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17142-52c0-4157-90f7-d3f0c1970bfe None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:48.741602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17142-52c0-4157-90f7-d3f0c1970bfe None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:48.763110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:48.778338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e2dbc-0923-47a4-af6f-d381372fac85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:48.778978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e2dbc-0923-47a4-af6f-d381372fac85 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:48.779253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e2dbc-0923-47a4-af6f-d381372fac85 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:48.779522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e2dbc-0923-47a4-af6f-d381372fac85 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:48.780195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e2dbc-0923-47a4-af6f-d381372fac85 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:48.781740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e2dbc-0923-47a4-af6f-d381372fac85 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:48.813085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 27/57] 10.222.0.75 () {60 vars in 1140 bytes} [Fri Sep 23 15:03:48 2022] POST /identity/v3/projects => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:03:50.965257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46782b5-90fa-4b9c-986d-49aa5de53ae7 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:50.965810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46782b5-90fa-4b9c-986d-49aa5de53ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:50.965810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46782b5-90fa-4b9c-986d-49aa5de53ae7 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:50.967140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 31/58] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:50.972735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebd1bf-2731-4d90-9eb9-6661d7bdb052 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:50.973163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebd1bf-2731-4d90-9eb9-6661d7bdb052 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:50.973248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebd1bf-2731-4d90-9eb9-6661d7bdb052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:50.992803 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-84ebd1bf-2731-4d90-9eb9-6661d7bdb052 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:50.993994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-84ebd1bf-2731-4d90-9eb9-6661d7bdb052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:51.066320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 28/59] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:03:51.335536 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46782b5-90fa-4b9c-986d-49aa5de53ae7 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:51.443863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46782b5-90fa-4b9c-986d-49aa5de53ae7 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:51.479271 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b56fd-faad-470b-9ed6-bb0d3e2d58ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:51.479943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b56fd-faad-470b-9ed6-bb0d3e2d58ac None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:51.480437 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b56fd-faad-470b-9ed6-bb0d3e2d58ac None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:51.480437 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b56fd-faad-470b-9ed6-bb0d3e2d58ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:51.481617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b56fd-faad-470b-9ed6-bb0d3e2d58ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:51.483082 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b56fd-faad-470b-9ed6-bb0d3e2d58ac None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:51.484991 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 32/60] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:51.491709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebd1bf-2731-4d90-9eb9-6661d7bdb052 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:51.599810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebd1bf-2731-4d90-9eb9-6661d7bdb052 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:51.601261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:51.639691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be920f-3f90-4604-a30d-eec2660abe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:51.640304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be920f-3f90-4604-a30d-eec2660abe4e None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:51.640580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be920f-3f90-4604-a30d-eec2660abe4e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:51.640884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be920f-3f90-4604-a30d-eec2660abe4e None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:51.641549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be920f-3f90-4604-a30d-eec2660abe4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:51.643013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be920f-3f90-4604-a30d-eec2660abe4e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:51.672815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 29/61] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:03:51 2022] POST /identity/v3/users => generated 275 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:03:55.173152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157241a1-0878-4a81-8e9e-67ece56a373e None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:55.174176 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157241a1-0878-4a81-8e9e-67ece56a373e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:55.174652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157241a1-0878-4a81-8e9e-67ece56a373e None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:55.176230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 33/62] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:55.182024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b04305-bcc7-4e52-b779-abc403e53d69 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:55.183139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b04305-bcc7-4e52-b779-abc403e53d69 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:55.183652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b04305-bcc7-4e52-b779-abc403e53d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:55.210509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-32b04305-bcc7-4e52-b779-abc403e53d69 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:55.212077 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-32b04305-bcc7-4e52-b779-abc403e53d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:55.308031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 30/63] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:03:55.394311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01aed95-3d91-4775-afa4-2d8abb931b43 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:55.394936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01aed95-3d91-4775-afa4-2d8abb931b43 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:55.395406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01aed95-3d91-4775-afa4-2d8abb931b43 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:55.397037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 34/64] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:55.412369 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e5dde-767f-4e3c-ba25-c29f0a7a0a30 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:55.413006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e5dde-767f-4e3c-ba25-c29f0a7a0a30 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:55.413498 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e5dde-767f-4e3c-ba25-c29f0a7a0a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:55.437490 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-3b0e5dde-767f-4e3c-ba25-c29f0a7a0a30 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:55.439010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3b0e5dde-767f-4e3c-ba25-c29f0a7a0a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:55.458057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d630b-ea7d-4e78-9b15-4319b2492d79 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:55.458680 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d630b-ea7d-4e78-9b15-4319b2492d79 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:55.459148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d630b-ea7d-4e78-9b15-4319b2492d79 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:55.460704 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 35/65] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:55.467319 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bcf27-8115-4f1a-9d2f-a6683bf8c3f6 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:55.467916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bcf27-8115-4f1a-9d2f-a6683bf8c3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:55.468183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bcf27-8115-4f1a-9d2f-a6683bf8c3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:55.488391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1f1bcf27-8115-4f1a-9d2f-a6683bf8c3f6 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:55.489973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1f1bcf27-8115-4f1a-9d2f-a6683bf8c3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:55.509505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:55.538971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 31/66] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:03:55.540623 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffef38-5725-4970-a13e-ab938279df6d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:55.546417 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffef38-5725-4970-a13e-ab938279df6d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:55.546895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffef38-5725-4970-a13e-ab938279df6d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:55.548563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 32/67] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:55.554086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5b2ef-3ae4-4ed5-8617-3efda6bd1859 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:55.554576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5b2ef-3ae4-4ed5-8617-3efda6bd1859 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:55.555064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5b2ef-3ae4-4ed5-8617-3efda6bd1859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:55.590102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b4f5b2ef-3ae4-4ed5-8617-3efda6bd1859 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:55.593274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b4f5b2ef-3ae4-4ed5-8617-3efda6bd1859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:55.597559 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 36/68] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:03:55.614844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:55.686742 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bcf27-8115-4f1a-9d2f-a6683bf8c3f6 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:55.690347 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 33/69] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:03:55.761464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6509f4a-2c18-4596-b794-3efaddc341f6 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:55.762071 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6509f4a-2c18-4596-b794-3efaddc341f6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:55.762333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6509f4a-2c18-4596-b794-3efaddc341f6 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:55.763901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 34/70] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:03:55.769861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fc36f-05d8-497d-9d87-8342dc37eb5f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:55.770401 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fc36f-05d8-497d-9d87-8342dc37eb5f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:55.770893 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fc36f-05d8-497d-9d87-8342dc37eb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:55.797948 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bcf27-8115-4f1a-9d2f-a6683bf8c3f6 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:55.806605 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c95fc36f-05d8-497d-9d87-8342dc37eb5f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:55.808008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c95fc36f-05d8-497d-9d87-8342dc37eb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:55.883863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aacc65-9515-4f2a-a759-e11e858c4cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:55.884514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aacc65-9515-4f2a-a759-e11e858c4cff None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:55.884823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aacc65-9515-4f2a-a759-e11e858c4cff None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:55.885096 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aacc65-9515-4f2a-a759-e11e858c4cff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:55.891215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 35/71] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:03:55.892465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fc36f-05d8-497d-9d87-8342dc37eb5f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:55.902889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aacc65-9515-4f2a-a759-e11e858c4cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-02aacc65-9515-4f2a-a759-e11e858c4cff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:03:55.908205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:55.909085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:55.909824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:55.909824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:55.909824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:55.909824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:55.909824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:03:55.909824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:03:55.909824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:03:55.909824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:03:55.909824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:03:55.909824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:03:55.909824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:03:55.909824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:03:55.912899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 37/72] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:03:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:03:55.925603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aacc65-9515-4f2a-a759-e11e858c4cff None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:55.948946 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:55.997187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:56.023713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fc36f-05d8-497d-9d87-8342dc37eb5f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:56.061965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06400dd-c8da-4d88-8a41-9f041182e3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:56.062607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06400dd-c8da-4d88-8a41-9f041182e3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:56.062884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06400dd-c8da-4d88-8a41-9f041182e3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:56.063156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06400dd-c8da-4d88-8a41-9f041182e3a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:56.074778 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06400dd-c8da-4d88-8a41-9f041182e3a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-b06400dd-c8da-4d88-8a41-9f041182e3a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:03:56.079864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:56.080944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:56.081805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:56.081805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:56.081805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:56.081805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:56.081805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:03:56.081805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:03:56.081805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:03:56.081805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:03:56.081805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:03:56.081805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:03:56.081805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:03:56.081805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:03:56.081805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 36/73] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:03:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:03:56.082410 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06400dd-c8da-4d88-8a41-9f041182e3a2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:56.100690 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aacc65-9515-4f2a-a759-e11e858c4cff None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:56.142736 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd2936-e842-4d5a-8195-c0bf9360e9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:56.143343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd2936-e842-4d5a-8195-c0bf9360e9cf None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:56.143613 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd2936-e842-4d5a-8195-c0bf9360e9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:56.143983 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd2936-e842-4d5a-8195-c0bf9360e9cf None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:56.144684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd2936-e842-4d5a-8195-c0bf9360e9cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:56.144984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd2936-e842-4d5a-8195-c0bf9360e9cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:56.146366 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd2936-e842-4d5a-8195-c0bf9360e9cf None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:56.154942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 38/74] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:03:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:56.155754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd2936-e842-4d5a-8195-c0bf9360e9cf None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:56.193561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06400dd-c8da-4d88-8a41-9f041182e3a2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:56.228538 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f34cef-6a1e-49a4-95d2-04608e890701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:56.229107 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f34cef-6a1e-49a4-95d2-04608e890701 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:56.229380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f34cef-6a1e-49a4-95d2-04608e890701 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:56.229701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f34cef-6a1e-49a4-95d2-04608e890701 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:56.231433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:56.237501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:56.242194 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f34cef-6a1e-49a4-95d2-04608e890701 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-20f34cef-6a1e-49a4-95d2-04608e890701 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:03:56.247101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:56.247938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:56.248705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:56.248705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:56.248705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:56.248705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:56.248705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:03:56.248705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:03:56.248705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:03:56.248705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:03:56.248705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:03:56.248705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:03:56.248705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:03:56.248705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:03:56.248705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 37/75] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:03:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:56.249335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f34cef-6a1e-49a4-95d2-04608e890701 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:56.266455 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd2936-e842-4d5a-8195-c0bf9360e9cf None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:56.299750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cd9e4-bc77-4646-a5af-8c9a9dc86d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:56.300385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cd9e4-bc77-4646-a5af-8c9a9dc86d9e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:56.300689 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cd9e4-bc77-4646-a5af-8c9a9dc86d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:56.300952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cd9e4-bc77-4646-a5af-8c9a9dc86d9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:56.311569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621cd9e4-bc77-4646-a5af-8c9a9dc86d9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-621cd9e4-bc77-4646-a5af-8c9a9dc86d9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:03:56.317115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:03:56.318099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 39/76] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:03:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:03:56.319029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cd9e4-bc77-4646-a5af-8c9a9dc86d9e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:56.355439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f34cef-6a1e-49a4-95d2-04608e890701 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:56.391301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a12804-9a78-4ddb-a120-c128feec3da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:56.391900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a12804-9a78-4ddb-a120-c128feec3da5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:56.392228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a12804-9a78-4ddb-a120-c128feec3da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:56.392497 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a12804-9a78-4ddb-a120-c128feec3da5 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:56.393224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a12804-9a78-4ddb-a120-c128feec3da5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:56.393519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a12804-9a78-4ddb-a120-c128feec3da5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:56.394911 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a12804-9a78-4ddb-a120-c128feec3da5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:56.403840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 38/77] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:03:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:56.404950 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a12804-9a78-4ddb-a120-c128feec3da5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:56.424344 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cd9e4-bc77-4646-a5af-8c9a9dc86d9e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:56.453721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:56.459344 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080291c6-c659-4708-8b62-89b7aea2788d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:56.459986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080291c6-c659-4708-8b62-89b7aea2788d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:56.460259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080291c6-c659-4708-8b62-89b7aea2788d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:56.460531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080291c6-c659-4708-8b62-89b7aea2788d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:56.461702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080291c6-c659-4708-8b62-89b7aea2788d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:56.463161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080291c6-c659-4708-8b62-89b7aea2788d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:56.465477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 40/78] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:03:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:56.466036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080291c6-c659-4708-8b62-89b7aea2788d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:56.473124 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:56.515646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a12804-9a78-4ddb-a120-c128feec3da5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:56.550961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9f095-f63f-4538-979a-089d36eb1564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:56.551606 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9f095-f63f-4538-979a-089d36eb1564 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:56.551885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9f095-f63f-4538-979a-089d36eb1564 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:56.552157 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9f095-f63f-4538-979a-089d36eb1564 None admin] PATH_INFO: `/v3/users/4cdec6738f534b16af19e42f8f462d49` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:56.569213 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9f095-f63f-4538-979a-089d36eb1564 None admin] RBAC: Authorizing `identity:get_user(user_id=4cdec6738f534b16af19e42f8f462d49)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:56.571102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9f095-f63f-4538-979a-089d36eb1564 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:56.571813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080291c6-c659-4708-8b62-89b7aea2788d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:56.573165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 39/79] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:03:56 2022] GET /identity/v3/users/4cdec6738f534b16af19e42f8f462d49 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:56.574125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9f095-f63f-4538-979a-089d36eb1564 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:56.606930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77dd01-3b23-4296-bf77-572e47dba4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:56.607548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77dd01-3b23-4296-bf77-572e47dba4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:56.607849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77dd01-3b23-4296-bf77-572e47dba4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:56.608122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77dd01-3b23-4296-bf77-572e47dba4d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:56.609129 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77dd01-3b23-4296-bf77-572e47dba4d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:56.609434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77dd01-3b23-4296-bf77-572e47dba4d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:56.610863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77dd01-3b23-4296-bf77-572e47dba4d2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:56.619113 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 41/80] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:03:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:56.624756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77dd01-3b23-4296-bf77-572e47dba4d2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:56.678215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9f095-f63f-4538-979a-089d36eb1564 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:56.689039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:56.697516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:56.715916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a85c63-c5ba-43ce-9e64-f5e76cc169b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:56.716525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a85c63-c5ba-43ce-9e64-f5e76cc169b5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:56.716874 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a85c63-c5ba-43ce-9e64-f5e76cc169b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:56.717108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a85c63-c5ba-43ce-9e64-f5e76cc169b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:56.717780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a85c63-c5ba-43ce-9e64-f5e76cc169b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:56.718077 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a85c63-c5ba-43ce-9e64-f5e76cc169b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:56.719467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a85c63-c5ba-43ce-9e64-f5e76cc169b5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:56.728013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 40/81] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:03:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:56.728822 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a85c63-c5ba-43ce-9e64-f5e76cc169b5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:56.729012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77dd01-3b23-4296-bf77-572e47dba4d2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:56.764962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b11ba-1c0c-4fb2-9abc-0fb8107ce2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:56.765606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b11ba-1c0c-4fb2-9abc-0fb8107ce2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:56.765938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b11ba-1c0c-4fb2-9abc-0fb8107ce2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:56.766159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b11ba-1c0c-4fb2-9abc-0fb8107ce2e4 None admin] PATH_INFO: `/v3/users/4cdec6738f534b16af19e42f8f462d49` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:56.767317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b11ba-1c0c-4fb2-9abc-0fb8107ce2e4 None admin] RBAC: Authorizing `identity:get_user(user_id=4cdec6738f534b16af19e42f8f462d49)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:56.768805 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b11ba-1c0c-4fb2-9abc-0fb8107ce2e4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:56.770615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 42/82] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:03:56 2022] GET /identity/v3/users/4cdec6738f534b16af19e42f8f462d49 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:56.771249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b11ba-1c0c-4fb2-9abc-0fb8107ce2e4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:56.834687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a85c63-c5ba-43ce-9e64-f5e76cc169b5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:56.868695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39e71b-727c-407b-b880-d95b838719e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:56.869314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39e71b-727c-407b-b880-d95b838719e2 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:56.869592 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39e71b-727c-407b-b880-d95b838719e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:56.869858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa39e71b-727c-407b-b880-d95b838719e2 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:56.870742 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39e71b-727c-407b-b880-d95b838719e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:56.872234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa39e71b-727c-407b-b880-d95b838719e2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:56.879206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b11ba-1c0c-4fb2-9abc-0fb8107ce2e4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:56.904690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 41/83] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:03:56 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:03:56.904690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39e71b-727c-407b-b880-d95b838719e2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:56.919756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:56.924009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463ff2c-55ef-437d-bc43-320b0ca0a6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:56.924652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463ff2c-55ef-437d-bc43-320b0ca0a6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:56.925016 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463ff2c-55ef-437d-bc43-320b0ca0a6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:56.925286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463ff2c-55ef-437d-bc43-320b0ca0a6d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:56.946301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:57.009260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463ff2c-55ef-437d-bc43-320b0ca0a6d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:57.011834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463ff2c-55ef-437d-bc43-320b0ca0a6d9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:57.041046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa39e71b-727c-407b-b880-d95b838719e2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:57.080580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f5d33-422a-4fdf-90ac-7aea192c46c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:57.081263 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f5d33-422a-4fdf-90ac-7aea192c46c8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:57.081538 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f5d33-422a-4fdf-90ac-7aea192c46c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:57.081801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83f5d33-422a-4fdf-90ac-7aea192c46c8 None admin] PATH_INFO: `/v3/users/1d564d82faf2482faf5205415837bf68` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:57.083009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f5d33-422a-4fdf-90ac-7aea192c46c8 None admin] RBAC: Authorizing `identity:get_user(user_id=1d564d82faf2482faf5205415837bf68)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:57.084513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83f5d33-422a-4fdf-90ac-7aea192c46c8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:57.086304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 42/84] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:03:56 2022] GET /identity/v3/users/1d564d82faf2482faf5205415837bf68 => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:57.090305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f5d33-422a-4fdf-90ac-7aea192c46c8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:57.161186 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 43/85] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:03:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:03:57.162040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463ff2c-55ef-437d-bc43-320b0ca0a6d9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:57.195439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:57.223492 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83f5d33-422a-4fdf-90ac-7aea192c46c8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:57.226579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:57.265442 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfec83c-1658-423e-994b-55f60bfb4cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:57.266654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfec83c-1658-423e-994b-55f60bfb4cda None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:57.266654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfec83c-1658-423e-994b-55f60bfb4cda None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:57.266654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfec83c-1658-423e-994b-55f60bfb4cda None admin] PATH_INFO: `/v3/users/4cdec6738f534b16af19e42f8f462d49` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:57.268482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfec83c-1658-423e-994b-55f60bfb4cda None admin] RBAC: Authorizing `identity:get_user(user_id=4cdec6738f534b16af19e42f8f462d49)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:57.269991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfec83c-1658-423e-994b-55f60bfb4cda None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:57.271605 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 43/86] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:03:57 2022] GET /identity/v3/users/4cdec6738f534b16af19e42f8f462d49 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:57.272380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfec83c-1658-423e-994b-55f60bfb4cda None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:57.289127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463ff2c-55ef-437d-bc43-320b0ca0a6d9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:57.332788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47451e7a-98c0-4ddd-87ec-eac036e48e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:57.333410 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47451e7a-98c0-4ddd-87ec-eac036e48e79 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:57.333686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47451e7a-98c0-4ddd-87ec-eac036e48e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:57.333957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47451e7a-98c0-4ddd-87ec-eac036e48e79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:57.391969 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfec83c-1658-423e-994b-55f60bfb4cda None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:57.427745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47451e7a-98c0-4ddd-87ec-eac036e48e79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:57.429453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47451e7a-98c0-4ddd-87ec-eac036e48e79 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:57.429847 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ca9f5-8e03-4b1f-818a-d3d8b0c73f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:57.430459 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ca9f5-8e03-4b1f-818a-d3d8b0c73f39 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:57.430733 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ca9f5-8e03-4b1f-818a-d3d8b0c73f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:57.431005 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ca9f5-8e03-4b1f-818a-d3d8b0c73f39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:57.468184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:57.487742 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:57.522134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ca9f5-8e03-4b1f-818a-d3d8b0c73f39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:57.523813 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ca9f5-8e03-4b1f-818a-d3d8b0c73f39 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:57.564884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 44/87] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:03:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:03:57.570073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47451e7a-98c0-4ddd-87ec-eac036e48e79 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:57.667262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 44/88] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:03:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:03:57.668894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ca9f5-8e03-4b1f-818a-d3d8b0c73f39 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:57.692245 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47451e7a-98c0-4ddd-87ec-eac036e48e79 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:57.735417 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:57.745399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4b782-0151-4cf2-87e0-114f24e3b6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:57.746959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4b782-0151-4cf2-87e0-114f24e3b6dc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:57.747387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4b782-0151-4cf2-87e0-114f24e3b6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:57.747800 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4b782-0151-4cf2-87e0-114f24e3b6dc None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:57.749409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4b782-0151-4cf2-87e0-114f24e3b6dc None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:57.751524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:57.753338 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4b782-0151-4cf2-87e0-114f24e3b6dc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:57.755381 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 45/89] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:03:57 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:57.756293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4b782-0151-4cf2-87e0-114f24e3b6dc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:57.784855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ca9f5-8e03-4b1f-818a-d3d8b0c73f39 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:57.825802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480c07e-54e4-4e5a-834d-3aa32c7e010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:57.826634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480c07e-54e4-4e5a-834d-3aa32c7e010c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:57.827057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480c07e-54e4-4e5a-834d-3aa32c7e010c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:57.827429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480c07e-54e4-4e5a-834d-3aa32c7e010c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:57.907457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4b782-0151-4cf2-87e0-114f24e3b6dc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:57.910730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480c07e-54e4-4e5a-834d-3aa32c7e010c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:57.913204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480c07e-54e4-4e5a-834d-3aa32c7e010c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:57.960170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7948e5-e5d0-4f79-86ee-95b561399e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:57.961097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7948e5-e5d0-4f79-86ee-95b561399e62 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:57.961552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7948e5-e5d0-4f79-86ee-95b561399e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:57.962513 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7948e5-e5d0-4f79-86ee-95b561399e62 None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:57.963990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7948e5-e5d0-4f79-86ee-95b561399e62 None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:57.966664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7948e5-e5d0-4f79-86ee-95b561399e62 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:57.969076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 46/90] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:03:57 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:57.970491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7948e5-e5d0-4f79-86ee-95b561399e62 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:58.004070 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:58.046785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 45/91] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:03:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:03:58.047881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480c07e-54e4-4e5a-834d-3aa32c7e010c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:58.066388 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:58.105981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7948e5-e5d0-4f79-86ee-95b561399e62 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:58.141244 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d876dd-d1e0-440d-a06e-c39b2604ec48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:58.141866 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d876dd-d1e0-440d-a06e-c39b2604ec48 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:58.142139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d876dd-d1e0-440d-a06e-c39b2604ec48 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:58.142407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d876dd-d1e0-440d-a06e-c39b2604ec48 None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:58.143551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d876dd-d1e0-440d-a06e-c39b2604ec48 None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:58.145087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d876dd-d1e0-440d-a06e-c39b2604ec48 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:58.146953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 47/92] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:03:57 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:58.147601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d876dd-d1e0-440d-a06e-c39b2604ec48 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:58.177978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480c07e-54e4-4e5a-834d-3aa32c7e010c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:58.218774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf6af1-e610-4766-a29b-082d8524d5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:58.219325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf6af1-e610-4766-a29b-082d8524d5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:58.219602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf6af1-e610-4766-a29b-082d8524d5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:58.219912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf6af1-e610-4766-a29b-082d8524d5c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:58.220673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf6af1-e610-4766-a29b-082d8524d5c3 None admin] RBAC: Adding query filter params (role.id=704c67b455b54fb48089faef4aba8ccd, scope.project.id=9edb599ff8bb4c67b467b712ea2dc700, user.id=4cdec6738f534b16af19e42f8f462d49) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:58.220981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf6af1-e610-4766-a29b-082d8524d5c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:58.222592 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf6af1-e610-4766-a29b-082d8524d5c3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:58.229445 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 46/93] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:03:58 2022] GET /identity/v3/role_assignments?user.id=4cdec6738f534b16af19e42f8f462d49&scope.project.id=9edb599ff8bb4c67b467b712ea2dc700&role.id=704c67b455b54fb48089faef4aba8ccd => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:58.230685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf6af1-e610-4766-a29b-082d8524d5c3 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:58.275443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:58.294498 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d876dd-d1e0-440d-a06e-c39b2604ec48 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:58.379024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ca9cd-41ea-47bc-8660-b6979eb9976a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:58.379654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ca9cd-41ea-47bc-8660-b6979eb9976a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:58.379925 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ca9cd-41ea-47bc-8660-b6979eb9976a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:58.380189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ca9cd-41ea-47bc-8660-b6979eb9976a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:58.381056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ca9cd-41ea-47bc-8660-b6979eb9976a None admin] RBAC: Adding query filter params (role.id=2921a7146fd9408396ca98a4be4ecd3f, scope.project.id=9edb599ff8bb4c67b467b712ea2dc700, user.id=4cdec6738f534b16af19e42f8f462d49) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:58.381435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ca9cd-41ea-47bc-8660-b6979eb9976a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:58.383291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ca9cd-41ea-47bc-8660-b6979eb9976a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:58.387779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:58.396468 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 48/94] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:03:58 2022] GET /identity/v3/role_assignments?user.id=4cdec6738f534b16af19e42f8f462d49&scope.project.id=9edb599ff8bb4c67b467b712ea2dc700&role.id=2921a7146fd9408396ca98a4be4ecd3f => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:58.397394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ca9cd-41ea-47bc-8660-b6979eb9976a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:58.417420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf6af1-e610-4766-a29b-082d8524d5c3 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:58.472693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c79b2a-c40b-4ae8-a78f-4fe7feae7351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:58.473293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c79b2a-c40b-4ae8-a78f-4fe7feae7351 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:58.473618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c79b2a-c40b-4ae8-a78f-4fe7feae7351 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:58.473893 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c79b2a-c40b-4ae8-a78f-4fe7feae7351 None admin] PATH_INFO: `/v3/projects/84e33566ed254f9fb0adf234a6d5ec15` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:58.475083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c79b2a-c40b-4ae8-a78f-4fe7feae7351 None admin] RBAC: Authorizing `identity:get_project(project_id=84e33566ed254f9fb0adf234a6d5ec15)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:58.476728 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c79b2a-c40b-4ae8-a78f-4fe7feae7351 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:58.478988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 47/95] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:03:58 2022] GET /identity/v3/projects/84e33566ed254f9fb0adf234a6d5ec15 => generated 312 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:58.489428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c79b2a-c40b-4ae8-a78f-4fe7feae7351 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:03:58.601585 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ca9cd-41ea-47bc-8660-b6979eb9976a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:58.658792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822539c7-e426-4fa6-bbae-95739dd24ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:58.659649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822539c7-e426-4fa6-bbae-95739dd24ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:58.659927 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822539c7-e426-4fa6-bbae-95739dd24ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:58.660196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822539c7-e426-4fa6-bbae-95739dd24ca4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:58.660966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822539c7-e426-4fa6-bbae-95739dd24ca4 None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=9edb599ff8bb4c67b467b712ea2dc700, user.id=1d564d82faf2482faf5205415837bf68) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:58.661268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822539c7-e426-4fa6-bbae-95739dd24ca4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:58.663602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822539c7-e426-4fa6-bbae-95739dd24ca4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:58.670154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 49/96] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:03:58 2022] GET /identity/v3/role_assignments?user.id=1d564d82faf2482faf5205415837bf68&scope.project.id=9edb599ff8bb4c67b467b712ea2dc700&role.id=ae444a489623434999d758b71ae2dc87 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:58.739591 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c79b2a-c40b-4ae8-a78f-4fe7feae7351 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:03:58.742473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:03:58.791368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362249cc-3673-4a90-85c0-99fa66ab570a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:03:58.792065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362249cc-3673-4a90-85c0-99fa66ab570a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:58.792356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362249cc-3673-4a90-85c0-99fa66ab570a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:58.792659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362249cc-3673-4a90-85c0-99fa66ab570a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:58.793432 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362249cc-3673-4a90-85c0-99fa66ab570a None admin] RBAC: Adding query filter params (role.id=2921a7146fd9408396ca98a4be4ecd3f, scope.project.id=84e33566ed254f9fb0adf234a6d5ec15, user.id=4cdec6738f534b16af19e42f8f462d49) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:03:58.793737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362249cc-3673-4a90-85c0-99fa66ab570a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:03:58.795501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362249cc-3673-4a90-85c0-99fa66ab570a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:03:58.805448 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 48/97] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:03:58 2022] GET /identity/v3/role_assignments?user.id=4cdec6738f534b16af19e42f8f462d49&scope.project.id=84e33566ed254f9fb0adf234a6d5ec15&role.id=2921a7146fd9408396ca98a4be4ecd3f => generated 259 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:03:59.828540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3e52a-851a-4cfd-86c6-6320ed6223ce None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:59.829111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3e52a-851a-4cfd-86c6-6320ed6223ce None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:59.829111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3e52a-851a-4cfd-86c6-6320ed6223ce None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:59.830584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 50/98] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:03:59.836073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223a231-dc20-4ba0-8f78-99d68a29f633 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:59.836491 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223a231-dc20-4ba0-8f78-99d68a29f633 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:59.836705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223a231-dc20-4ba0-8f78-99d68a29f633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:59.859501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a223a231-dc20-4ba0-8f78-99d68a29f633 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:59.860779 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a223a231-dc20-4ba0-8f78-99d68a29f633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:03:59.872135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac178ab6-774b-404a-96d6-af6a40da896d None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:59.872409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac178ab6-774b-404a-96d6-af6a40da896d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:59.872710 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac178ab6-774b-404a-96d6-af6a40da896d None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:59.874056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 51/99] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:03:59.879477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaae68-dc4f-4f81-8977-4b8dfc392b0b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:03:59.879756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaae68-dc4f-4f81-8977-4b8dfc392b0b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:03:59.880030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaae68-dc4f-4f81-8977-4b8dfc392b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:03:59.910766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9daaae68-dc4f-4f81-8977-4b8dfc392b0b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:03:59.911974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9daaae68-dc4f-4f81-8977-4b8dfc392b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:00.004819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 49/100] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:00.020224 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:00.036360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 52/101] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:00.350514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223a231-dc20-4ba0-8f78-99d68a29f633 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:00.464808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:00.469204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223a231-dc20-4ba0-8f78-99d68a29f633 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:00.505195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23016451-f912-460d-8bad-f26eae82dca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:00.506172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23016451-f912-460d-8bad-f26eae82dca1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:00.506641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23016451-f912-460d-8bad-f26eae82dca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:00.507129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23016451-f912-460d-8bad-f26eae82dca1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:00.517999 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23016451-f912-460d-8bad-f26eae82dca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-23016451-f912-460d-8bad-f26eae82dca1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:00.523212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:00.524057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:00.524823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:00.524823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:00.524823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:00.524823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:00.524823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:00.524823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:00.524823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:00.524823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:00.524823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:00.524823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:00.524823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:00.524823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:00.527616 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 50/102] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:00.533588 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daaae68-dc4f-4f81-8977-4b8dfc392b0b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:00.545051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23016451-f912-460d-8bad-f26eae82dca1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:00.722260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daaae68-dc4f-4f81-8977-4b8dfc392b0b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:00.803017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6310e-8b52-4091-98a8-f3143a0b9ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:00.804002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6310e-8b52-4091-98a8-f3143a0b9ffc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:00.804428 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6310e-8b52-4091-98a8-f3143a0b9ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:00.804860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6310e-8b52-4091-98a8-f3143a0b9ffc None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:00.805689 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6310e-8b52-4091-98a8-f3143a0b9ffc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:00.806179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6310e-8b52-4091-98a8-f3143a0b9ffc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:00.807896 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6310e-8b52-4091-98a8-f3143a0b9ffc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:00.827768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23016451-f912-460d-8bad-f26eae82dca1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:00.829497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 53/103] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:00.835379 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6310e-8b52-4091-98a8-f3143a0b9ffc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:00.876903 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaae42-4229-44c9-8b2a-f07454f40a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:00.878271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaae42-4229-44c9-8b2a-f07454f40a82 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:00.878744 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaae42-4229-44c9-8b2a-f07454f40a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:00.879207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaae42-4229-44c9-8b2a-f07454f40a82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:00.880545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaae42-4229-44c9-8b2a-f07454f40a82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:00.882216 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaae42-4229-44c9-8b2a-f07454f40a82 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:00.884384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 51/104] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:00.890088 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaae42-4229-44c9-8b2a-f07454f40a82 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:00.936981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:00.946278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:01.043798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaae42-4229-44c9-8b2a-f07454f40a82 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:01.102634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6310e-8b52-4091-98a8-f3143a0b9ffc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:01.122673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56b163-f0db-4d3a-bf1a-3a18546a360e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:01.123604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56b163-f0db-4d3a-bf1a-3a18546a360e None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:01.124028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56b163-f0db-4d3a-bf1a-3a18546a360e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:01.124387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56b163-f0db-4d3a-bf1a-3a18546a360e None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:01.125296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56b163-f0db-4d3a-bf1a-3a18546a360e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:01.127373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa56b163-f0db-4d3a-bf1a-3a18546a360e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:01.175597 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 52/105] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:04:00 2022] POST /identity/v3/projects => generated 302 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:04:01.191479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33fb4d-ab96-4da3-b0ef-84eaa37c781f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:01.192138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33fb4d-ab96-4da3-b0ef-84eaa37c781f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:01.192413 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33fb4d-ab96-4da3-b0ef-84eaa37c781f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:01.192717 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33fb4d-ab96-4da3-b0ef-84eaa37c781f None admin] PATH_INFO: `/v3/users/2ffc232160674b439f3975bfd4232051` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:01.219566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33fb4d-ab96-4da3-b0ef-84eaa37c781f None admin] RBAC: Authorizing `identity:get_user(user_id=2ffc232160674b439f3975bfd4232051)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:01.221073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33fb4d-ab96-4da3-b0ef-84eaa37c781f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:01.223098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 54/106] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:00 2022] GET /identity/v3/users/2ffc232160674b439f3975bfd4232051 => generated 289 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:01.228870 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56b163-f0db-4d3a-bf1a-3a18546a360e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:01.305220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f859b-da1d-4fc7-9659-251998ab5523 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:01.305387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f859b-da1d-4fc7-9659-251998ab5523 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:01.305657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f859b-da1d-4fc7-9659-251998ab5523 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:01.307296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 55/107] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:01.335471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976eeaf-ffd7-4480-88f4-03b3035a9ebf None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:01.335812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976eeaf-ffd7-4480-88f4-03b3035a9ebf None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:01.336011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976eeaf-ffd7-4480-88f4-03b3035a9ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:01.368192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:01.371526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9976eeaf-ffd7-4480-88f4-03b3035a9ebf None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:01.372734 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9976eeaf-ffd7-4480-88f4-03b3035a9ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:01.418124 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56b163-f0db-4d3a-bf1a-3a18546a360e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:01.507242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba23a73-05b3-433d-b9bf-20d18d7f24a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:01.511782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba23a73-05b3-433d-b9bf-20d18d7f24a8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:01.512094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba23a73-05b3-433d-b9bf-20d18d7f24a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:01.512362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba23a73-05b3-433d-b9bf-20d18d7f24a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:01.554220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 56/108] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:01.623368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141277b-f610-415c-9b49-d6e8e1704328 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:01.623685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141277b-f610-415c-9b49-d6e8e1704328 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:01.623924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141277b-f610-415c-9b49-d6e8e1704328 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:01.625367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 57/109] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:04:01.628003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba23a73-05b3-433d-b9bf-20d18d7f24a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:01.629522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba23a73-05b3-433d-b9bf-20d18d7f24a8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:01.630928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597faf8-b270-4c14-80d5-d08fbf77ad36 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:01.631206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597faf8-b270-4c14-80d5-d08fbf77ad36 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:01.631475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597faf8-b270-4c14-80d5-d08fbf77ad36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:01.663484 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9597faf8-b270-4c14-80d5-d08fbf77ad36 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:01.664793 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9597faf8-b270-4c14-80d5-d08fbf77ad36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:01.748969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:01.760974 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:01.797885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 53/110] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:01.803929 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba23a73-05b3-433d-b9bf-20d18d7f24a8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:01.821871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 58/111] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:01.940633 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba23a73-05b3-433d-b9bf-20d18d7f24a8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:01.981892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e40ae2-4a49-467e-b88e-8c682528a7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:01.982545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e40ae2-4a49-467e-b88e-8c682528a7bd None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:01.982843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e40ae2-4a49-467e-b88e-8c682528a7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:01.983101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e40ae2-4a49-467e-b88e-8c682528a7bd None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:01.984650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e40ae2-4a49-467e-b88e-8c682528a7bd None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:01.986122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e40ae2-4a49-467e-b88e-8c682528a7bd None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:01.987780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 54/112] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:01 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:01.993739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597faf8-b270-4c14-80d5-d08fbf77ad36 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:02.025557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf7b4b-495c-4e68-a7c4-7af1f1092f23 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:02.025894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf7b4b-495c-4e68-a7c4-7af1f1092f23 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:02.026756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf7b4b-495c-4e68-a7c4-7af1f1092f23 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:02.028507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 55/113] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:04:02.043663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdfc6f-6810-49ff-b513-d645c32721d1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:02.043968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdfc6f-6810-49ff-b513-d645c32721d1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:02.044519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdfc6f-6810-49ff-b513-d645c32721d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:02.066131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-77bdfc6f-6810-49ff-b513-d645c32721d1 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:02.067791 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-77bdfc6f-6810-49ff-b513-d645c32721d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:02.148804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:02.159846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597faf8-b270-4c14-80d5-d08fbf77ad36 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:02.183613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 56/114] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:02.188180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdfc6f-6810-49ff-b513-d645c32721d1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:02.228164 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a91c5-9d0c-4f90-b9d4-a1f892dfbb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:02.229157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a91c5-9d0c-4f90-b9d4-a1f892dfbb18 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:02.229877 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a91c5-9d0c-4f90-b9d4-a1f892dfbb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:02.231422 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a91c5-9d0c-4f90-b9d4-a1f892dfbb18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:02.233038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a91c5-9d0c-4f90-b9d4-a1f892dfbb18 None admin] RBAC: Adding query filter params (role.id=712114868a39430b80ec9c19cb91c947, scope.project.id=9edb599ff8bb4c67b467b712ea2dc700, user.id=2ffc232160674b439f3975bfd4232051) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:02.235382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a91c5-9d0c-4f90-b9d4-a1f892dfbb18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:02.237319 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a91c5-9d0c-4f90-b9d4-a1f892dfbb18 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:02.249980 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 59/115] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:01 2022] GET /identity/v3/role_assignments?user.id=2ffc232160674b439f3975bfd4232051&scope.project.id=9edb599ff8bb4c67b467b712ea2dc700&role.id=712114868a39430b80ec9c19cb91c947 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:02.253118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a91c5-9d0c-4f90-b9d4-a1f892dfbb18 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:02.336110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdfc6f-6810-49ff-b513-d645c32721d1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:02.336851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:02.377280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32935aa2-5be4-436a-8401-71df7db5bd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:02.377941 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32935aa2-5be4-436a-8401-71df7db5bd87 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:02.378214 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32935aa2-5be4-436a-8401-71df7db5bd87 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:02.378484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32935aa2-5be4-436a-8401-71df7db5bd87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:02.391423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32935aa2-5be4-436a-8401-71df7db5bd87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-32935aa2-5be4-436a-8401-71df7db5bd87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:02.396449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:02.397449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:02.398684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:02.398684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:02.398684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:02.398684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:02.398684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:02.398684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:02.398684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:02.398684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:02.398684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:02.398684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:02.398684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:02.398684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:02.398684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 57/116] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:02.399816 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f52f5e-ae3d-40cb-ba00-ca95f18d708d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:02.400099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f52f5e-ae3d-40cb-ba00-ca95f18d708d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:02.400379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f52f5e-ae3d-40cb-ba00-ca95f18d708d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:02.402786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 58/117] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:02.408732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f52f5e-ae3d-40cb-ba00-ca95f18d708d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:02.457325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:02.481457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a91c5-9d0c-4f90-b9d4-a1f892dfbb18 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:02.546234 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45b59e-c0eb-4be0-9c95-04097c85d0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:02.546953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45b59e-c0eb-4be0-9c95-04097c85d0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:02.547235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45b59e-c0eb-4be0-9c95-04097c85d0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:02.547870 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45b59e-c0eb-4be0-9c95-04097c85d0c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:02.563263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45b59e-c0eb-4be0-9c95-04097c85d0c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:02.566027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f52f5e-ae3d-40cb-ba00-ca95f18d708d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-6e45b59e-c0eb-4be0-9c95-04097c85d0c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:02.569746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:02.570607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:02.571779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:02.571779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:02.571779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:02.571779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:02.571779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:02.571779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:02.571779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:02.571779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:02.571779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:02.571779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:02.571779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:02.571779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:02.571779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 60/118] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:04:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:02.572864 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c59d-e6c8-4ce6-bfa0-fda9e0a99988 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:02.572864 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c59d-e6c8-4ce6-bfa0-fda9e0a99988 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:02.572864 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c59d-e6c8-4ce6-bfa0-fda9e0a99988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:02.593592 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2eb9c59d-e6c8-4ce6-bfa0-fda9e0a99988 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:02.595117 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2eb9c59d-e6c8-4ce6-bfa0-fda9e0a99988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:02.610750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ca156-3229-44d4-a1fa-c4ed15aecbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:02.611655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ca156-3229-44d4-a1fa-c4ed15aecbcf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:02.611930 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ca156-3229-44d4-a1fa-c4ed15aecbcf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:02.612203 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ca156-3229-44d4-a1fa-c4ed15aecbcf None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:02.613378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ca156-3229-44d4-a1fa-c4ed15aecbcf None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:02.613695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ca156-3229-44d4-a1fa-c4ed15aecbcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:02.615378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ca156-3229-44d4-a1fa-c4ed15aecbcf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:02.625829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 59/119] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:02.626819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ca156-3229-44d4-a1fa-c4ed15aecbcf None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:02.672547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 61/120] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:02.673929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9c59d-e6c8-4ce6-bfa0-fda9e0a99988 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:02.705430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:02.795866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:02.806230 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ca156-3229-44d4-a1fa-c4ed15aecbcf None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:02.868082 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9c59d-e6c8-4ce6-bfa0-fda9e0a99988 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:02.887923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e542b5-d333-4918-add1-07ac974bb7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:02.889026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e542b5-d333-4918-add1-07ac974bb7fa None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:02.889608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e542b5-d333-4918-add1-07ac974bb7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:02.890154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e542b5-d333-4918-add1-07ac974bb7fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:02.922909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e542b5-d333-4918-add1-07ac974bb7fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-55e542b5-d333-4918-add1-07ac974bb7fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:02.930958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:02.931977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:02.933064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:02.933064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:02.933064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:02.933064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:02.933064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:02.933064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:02.933064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:02.933064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:02.933064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:02.933064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:02.933064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:02.933064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:02.944267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 60/121] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:04:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:02.946137 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e542b5-d333-4918-add1-07ac974bb7fa None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:02.959722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b05e2c-9ed6-435b-87e6-ac84c0280850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:02.960528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b05e2c-9ed6-435b-87e6-ac84c0280850 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:02.960847 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b05e2c-9ed6-435b-87e6-ac84c0280850 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:02.961175 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b05e2c-9ed6-435b-87e6-ac84c0280850 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:02.962403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b05e2c-9ed6-435b-87e6-ac84c0280850 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:02.962697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b05e2c-9ed6-435b-87e6-ac84c0280850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:02.964721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b05e2c-9ed6-435b-87e6-ac84c0280850 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:02.976071 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 62/122] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:04:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:02.977261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b05e2c-9ed6-435b-87e6-ac84c0280850 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:03.101431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:03.117261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b05e2c-9ed6-435b-87e6-ac84c0280850 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:03.142558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e542b5-d333-4918-add1-07ac974bb7fa None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:03.177711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb8ece-2dc3-460d-8360-b4aa257f93e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:03.179113 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb8ece-2dc3-460d-8360-b4aa257f93e7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:03.179601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb8ece-2dc3-460d-8360-b4aa257f93e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:03.180273 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb8ece-2dc3-460d-8360-b4aa257f93e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:03.181317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb8ece-2dc3-460d-8360-b4aa257f93e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:03.181863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb8ece-2dc3-460d-8360-b4aa257f93e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:03.184092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb8ece-2dc3-460d-8360-b4aa257f93e7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:03.185035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096705ab-bbe2-4053-a9f6-a26789db5ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:03.185862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096705ab-bbe2-4053-a9f6-a26789db5ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:03.186232 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096705ab-bbe2-4053-a9f6-a26789db5ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:03.186596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096705ab-bbe2-4053-a9f6-a26789db5ea6 None admin] PATH_INFO: `/v3/users/4cdec6738f534b16af19e42f8f462d49` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:03.188209 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096705ab-bbe2-4053-a9f6-a26789db5ea6 None admin] RBAC: Authorizing `identity:get_user(user_id=4cdec6738f534b16af19e42f8f462d49)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:03.190634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096705ab-bbe2-4053-a9f6-a26789db5ea6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:03.192566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 61/123] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:02 2022] GET /identity/v3/users/4cdec6738f534b16af19e42f8f462d49 => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:03.197280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096705ab-bbe2-4053-a9f6-a26789db5ea6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:03.204162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 63/124] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:03.205079 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb8ece-2dc3-460d-8360-b4aa257f93e7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:03.240806 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:03.347393 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096705ab-bbe2-4053-a9f6-a26789db5ea6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:03.358719 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb8ece-2dc3-460d-8360-b4aa257f93e7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:03.401212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50f5f0-93e1-49ec-b56d-b1d4a904236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:03.402105 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50f5f0-93e1-49ec-b56d-b1d4a904236c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:03.402572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50f5f0-93e1-49ec-b56d-b1d4a904236c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:03.402871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50f5f0-93e1-49ec-b56d-b1d4a904236c None admin] PATH_INFO: `/v3/users/4cdec6738f534b16af19e42f8f462d49` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:03.404119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50f5f0-93e1-49ec-b56d-b1d4a904236c None admin] RBAC: Authorizing `identity:get_user(user_id=4cdec6738f534b16af19e42f8f462d49)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:03.405448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d293e4a-814c-4dc1-8b1c-57dc911df648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:03.405900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50f5f0-93e1-49ec-b56d-b1d4a904236c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:03.406802 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d293e4a-814c-4dc1-8b1c-57dc911df648 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:03.407344 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d293e4a-814c-4dc1-8b1c-57dc911df648 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:03.407899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d293e4a-814c-4dc1-8b1c-57dc911df648 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:03.412641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 62/125] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:03 2022] GET /identity/v3/users/4cdec6738f534b16af19e42f8f462d49 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:03.413641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50f5f0-93e1-49ec-b56d-b1d4a904236c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:03.427985 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d293e4a-814c-4dc1-8b1c-57dc911df648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-6d293e4a-814c-4dc1-8b1c-57dc911df648 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:03.435091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:03.436730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:03.437952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:03.437952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:03.437952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:03.437952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:03.437952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:03.437952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:03.437952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:03.437952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:03.437952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:03.437952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:03.437952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:03.437952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:03.440034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 64/126] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:03.446481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d293e4a-814c-4dc1-8b1c-57dc911df648 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:03.461473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:03.537643 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:03.548290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50f5f0-93e1-49ec-b56d-b1d4a904236c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:03.586545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d293e4a-814c-4dc1-8b1c-57dc911df648 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:03.597473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501f269-b274-4cd0-b9b9-f2a0a1f98fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:03.598530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501f269-b274-4cd0-b9b9-f2a0a1f98fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:03.599024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501f269-b274-4cd0-b9b9-f2a0a1f98fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:03.599562 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501f269-b274-4cd0-b9b9-f2a0a1f98fa1 None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:03.601123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501f269-b274-4cd0-b9b9-f2a0a1f98fa1 None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:03.603307 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501f269-b274-4cd0-b9b9-f2a0a1f98fa1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:03.606670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 63/127] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:03 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:03.608614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501f269-b274-4cd0-b9b9-f2a0a1f98fa1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:03.627005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c286a-1cae-4c9c-9f4c-915f597f8713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:03.627568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c286a-1cae-4c9c-9f4c-915f597f8713 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:03.627901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c286a-1cae-4c9c-9f4c-915f597f8713 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:03.628175 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c286a-1cae-4c9c-9f4c-915f597f8713 None admin] PATH_INFO: `/v3/users/1d564d82faf2482faf5205415837bf68` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:03.629371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c286a-1cae-4c9c-9f4c-915f597f8713 None admin] RBAC: Authorizing `identity:get_user(user_id=1d564d82faf2482faf5205415837bf68)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:03.631155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c286a-1cae-4c9c-9f4c-915f597f8713 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:03.632615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 65/128] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:03 2022] GET /identity/v3/users/1d564d82faf2482faf5205415837bf68 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:03.633389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c286a-1cae-4c9c-9f4c-915f597f8713 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:03.718886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:03.732555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501f269-b274-4cd0-b9b9-f2a0a1f98fa1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:03.745582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c286a-1cae-4c9c-9f4c-915f597f8713 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:03.777994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9402f-15d8-4ed7-865a-acc6a25ef847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:03.779165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9402f-15d8-4ed7-865a-acc6a25ef847 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:03.779704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9402f-15d8-4ed7-865a-acc6a25ef847 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:03.780161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9402f-15d8-4ed7-865a-acc6a25ef847 None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:03.781574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9402f-15d8-4ed7-865a-acc6a25ef847 None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:03.783422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9402f-15d8-4ed7-865a-acc6a25ef847 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:03.785332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 64/129] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:03 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:03.790009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b53475-08c7-44ed-96db-ccf7dd91c118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:03.790009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b53475-08c7-44ed-96db-ccf7dd91c118 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:03.790009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b53475-08c7-44ed-96db-ccf7dd91c118 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:03.790009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b53475-08c7-44ed-96db-ccf7dd91c118 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:03.790343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b53475-08c7-44ed-96db-ccf7dd91c118 None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:03.790343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b53475-08c7-44ed-96db-ccf7dd91c118 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:03.791335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b53475-08c7-44ed-96db-ccf7dd91c118 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:03.792156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9402f-15d8-4ed7-865a-acc6a25ef847 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:03.800960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 66/130] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:03.801952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b53475-08c7-44ed-96db-ccf7dd91c118 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:03.845618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:03.923551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b53475-08c7-44ed-96db-ccf7dd91c118 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:03.965058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e039cac-a699-4385-8db0-36aa5cf1a1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:03.965818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e039cac-a699-4385-8db0-36aa5cf1a1cc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:03.966149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e039cac-a699-4385-8db0-36aa5cf1a1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:03.966496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e039cac-a699-4385-8db0-36aa5cf1a1cc None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:03.967716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e039cac-a699-4385-8db0-36aa5cf1a1cc None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:03.968918 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9402f-15d8-4ed7-865a-acc6a25ef847 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:03.969564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e039cac-a699-4385-8db0-36aa5cf1a1cc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:03.971035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 67/131] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:03 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:03.971988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e039cac-a699-4385-8db0-36aa5cf1a1cc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:03.985656 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:04.032043 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18507b-8d93-44b4-aa23-0e6913292296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:04.033760 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18507b-8d93-44b4-aa23-0e6913292296 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:04.034539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18507b-8d93-44b4-aa23-0e6913292296 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:04.035310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18507b-8d93-44b4-aa23-0e6913292296 None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700/users/4cdec6738f534b16af19e42f8f462d49/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:04.038993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18507b-8d93-44b4-aa23-0e6913292296 None admin] RBAC: Authorizing `identity:create_grant(project_id=9edb599ff8bb4c67b467b712ea2dc700, user_id=4cdec6738f534b16af19e42f8f462d49, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:04.042358 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac18507b-8d93-44b4-aa23-0e6913292296 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:04.063338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 65/132] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:04:03 2022] PUT /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700/users/4cdec6738f534b16af19e42f8f462d49/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:04.064872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18507b-8d93-44b4-aa23-0e6913292296 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:04.094084 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e039cac-a699-4385-8db0-36aa5cf1a1cc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:04.134126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15937d3a-9489-426b-bfd8-f48d2a1c727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:04.134769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15937d3a-9489-426b-bfd8-f48d2a1c727d None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:04.134987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15937d3a-9489-426b-bfd8-f48d2a1c727d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:04.135227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15937d3a-9489-426b-bfd8-f48d2a1c727d None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700/users/4cdec6738f534b16af19e42f8f462d49/roles/704c67b455b54fb48089faef4aba8ccd` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:04.138146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15937d3a-9489-426b-bfd8-f48d2a1c727d None admin] RBAC: Authorizing `identity:create_grant(project_id=9edb599ff8bb4c67b467b712ea2dc700, user_id=4cdec6738f534b16af19e42f8f462d49, role_id=704c67b455b54fb48089faef4aba8ccd)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:04.139798 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15937d3a-9489-426b-bfd8-f48d2a1c727d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:04.160762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 68/133] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:04:03 2022] PUT /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700/users/4cdec6738f534b16af19e42f8f462d49/roles/704c67b455b54fb48089faef4aba8ccd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:04.161856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15937d3a-9489-426b-bfd8-f48d2a1c727d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:04.193849 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:04.209751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac18507b-8d93-44b4-aa23-0e6913292296 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:04.245599 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:04.281097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca60b69-d9cb-4b56-879c-6e367bc5abaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:04.282226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca60b69-d9cb-4b56-879c-6e367bc5abaa None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:04.282724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca60b69-d9cb-4b56-879c-6e367bc5abaa None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:04.283204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca60b69-d9cb-4b56-879c-6e367bc5abaa None admin] PATH_INFO: `/v3/users/4cdec6738f534b16af19e42f8f462d49` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:04.287860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca60b69-d9cb-4b56-879c-6e367bc5abaa None admin] RBAC: Authorizing `identity:get_user(user_id=4cdec6738f534b16af19e42f8f462d49)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:04.288029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15937d3a-9489-426b-bfd8-f48d2a1c727d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:04.289909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca60b69-d9cb-4b56-879c-6e367bc5abaa None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:04.292034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 66/134] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:04 2022] GET /identity/v3/users/4cdec6738f534b16af19e42f8f462d49 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:04.298116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca60b69-d9cb-4b56-879c-6e367bc5abaa None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:04.327187 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce1b2a-1aba-4f3a-bf5a-1c85350a6c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:04.327863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce1b2a-1aba-4f3a-bf5a-1c85350a6c21 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:04.328143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce1b2a-1aba-4f3a-bf5a-1c85350a6c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:04.328421 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce1b2a-1aba-4f3a-bf5a-1c85350a6c21 None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700/users/1d564d82faf2482faf5205415837bf68/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:04.330412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce1b2a-1aba-4f3a-bf5a-1c85350a6c21 None admin] RBAC: Authorizing `identity:create_grant(project_id=9edb599ff8bb4c67b467b712ea2dc700, user_id=1d564d82faf2482faf5205415837bf68, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:04.331892 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce1b2a-1aba-4f3a-bf5a-1c85350a6c21 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:04.352970 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 69/135] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:04:04 2022] PUT /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700/users/1d564d82faf2482faf5205415837bf68/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:04.377227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e22b3-66f7-4094-8128-9c3e7ee9c9d0 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:04.377562 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e22b3-66f7-4094-8128-9c3e7ee9c9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:04.377780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e22b3-66f7-4094-8128-9c3e7ee9c9d0 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:04.379360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 70/136] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:04:04.386286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599cf7c-c887-4868-a7e2-5f01935dd58a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:04.386612 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599cf7c-c887-4868-a7e2-5f01935dd58a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:04.386853 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c599cf7c-c887-4868-a7e2-5f01935dd58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:04.422465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c599cf7c-c887-4868-a7e2-5f01935dd58a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:04.430599 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c599cf7c-c887-4868-a7e2-5f01935dd58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:04.536957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca60b69-d9cb-4b56-879c-6e367bc5abaa None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:04.573390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 71/137] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 23 15:04:04.585549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616206e-5d18-4fe1-a641-54b485ba2be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:04.586615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616206e-5d18-4fe1-a641-54b485ba2be1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:04.586889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616206e-5d18-4fe1-a641-54b485ba2be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:04.587163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616206e-5d18-4fe1-a641-54b485ba2be1 None admin] PATH_INFO: `/v3/projects/84e33566ed254f9fb0adf234a6d5ec15` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:04.588462 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616206e-5d18-4fe1-a641-54b485ba2be1 None admin] RBAC: Authorizing `identity:get_project(project_id=84e33566ed254f9fb0adf234a6d5ec15)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:04.591161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616206e-5d18-4fe1-a641-54b485ba2be1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:04.593190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 67/138] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:04 2022] GET /identity/v3/projects/84e33566ed254f9fb0adf234a6d5ec15 => generated 312 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:04.598891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599cf7c-c887-4868-a7e2-5f01935dd58a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:04.677886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:04.735830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c599cf7c-c887-4868-a7e2-5f01935dd58a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:04.788247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9c8ce-d38c-4afa-b6a5-1a4b8bbb50a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:04.788928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c8ce-d38c-4afa-b6a5-1a4b8bbb50a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:04.789210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c8ce-d38c-4afa-b6a5-1a4b8bbb50a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:04.789487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9c8ce-d38c-4afa-b6a5-1a4b8bbb50a6 None admin] PATH_INFO: `/v3/projects/84e33566ed254f9fb0adf234a6d5ec15/users/4cdec6738f534b16af19e42f8f462d49/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:04.795449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9c8ce-d38c-4afa-b6a5-1a4b8bbb50a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=84e33566ed254f9fb0adf234a6d5ec15, user_id=4cdec6738f534b16af19e42f8f462d49, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:04.796983 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9c8ce-d38c-4afa-b6a5-1a4b8bbb50a6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:04.799871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed12bc-a891-47d6-8f09-eceb189d01e4 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:04.800152 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed12bc-a891-47d6-8f09-eceb189d01e4 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:04.800429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed12bc-a891-47d6-8f09-eceb189d01e4 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:04.801943 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 68/139] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:04:04.808540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0f4e-25b4-4451-969f-01e73e39304e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:04.808822 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0f4e-25b4-4451-969f-01e73e39304e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:04.809069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0f4e-25b4-4451-969f-01e73e39304e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:04.827169 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 72/140] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:04:04 2022] PUT /identity/v3/projects/84e33566ed254f9fb0adf234a6d5ec15/users/4cdec6738f534b16af19e42f8f462d49/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:04:04.841988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-65eb0f4e-25b4-4451-969f-01e73e39304e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:04.843250 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-65eb0f4e-25b4-4451-969f-01e73e39304e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:04.868907 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:04.921633 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 69/141] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:05.030688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9c8ce-d38c-4afa-b6a5-1a4b8bbb50a6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:05.158034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9c8ce-d38c-4afa-b6a5-1a4b8bbb50a6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:05.180742 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:05.203947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43765e-18e3-47af-80d7-c694e2084d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:05.204623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43765e-18e3-47af-80d7-c694e2084d82 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:05.204906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43765e-18e3-47af-80d7-c694e2084d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:05.205180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43765e-18e3-47af-80d7-c694e2084d82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:05.209336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43765e-18e3-47af-80d7-c694e2084d82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:05.210836 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43765e-18e3-47af-80d7-c694e2084d82 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:05.212488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 73/142] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:05.218282 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb0f4e-25b4-4451-969f-01e73e39304e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:05.231247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43765e-18e3-47af-80d7-c694e2084d82 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:05.410194 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb0f4e-25b4-4451-969f-01e73e39304e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:05.455358 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c00ef-d595-4dfb-bd34-7231f2894946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:05.456342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c00ef-d595-4dfb-bd34-7231f2894946 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:05.456509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43765e-18e3-47af-80d7-c694e2084d82 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:05.456990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c00ef-d595-4dfb-bd34-7231f2894946 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:05.457466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c00ef-d595-4dfb-bd34-7231f2894946 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:05.458604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c00ef-d595-4dfb-bd34-7231f2894946 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:05.460382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c00ef-d595-4dfb-bd34-7231f2894946 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:05.490999 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 70/143] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:04:05 2022] POST /identity/v3/users => generated 283 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:04:05.518605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc4511-0fec-477b-98ac-ac2eeb852658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:05.519796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc4511-0fec-477b-98ac-ac2eeb852658 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:05.520342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc4511-0fec-477b-98ac-ac2eeb852658 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:05.520832 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc4511-0fec-477b-98ac-ac2eeb852658 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:05.534237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc4511-0fec-477b-98ac-ac2eeb852658 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-66cc4511-0fec-477b-98ac-ac2eeb852658 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:05.539884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:05.540803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:05.541721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:05.541721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:05.541721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:05.541721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:05.541721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:05.541721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:05.541721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:05.541721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:05.541721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:05.541721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:05.541721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:05.541721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:05.544129 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 74/144] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:05.549778 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c00ef-d595-4dfb-bd34-7231f2894946 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:05.576784 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:05.683709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c00ef-d595-4dfb-bd34-7231f2894946 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:05.720354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b0a55-9b5e-44ee-b874-88a0e3e77998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:05.721007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b0a55-9b5e-44ee-b874-88a0e3e77998 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:05.721280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b0a55-9b5e-44ee-b874-88a0e3e77998 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:05.721546 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b0a55-9b5e-44ee-b874-88a0e3e77998 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:05.722227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b0a55-9b5e-44ee-b874-88a0e3e77998 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:05.722529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b0a55-9b5e-44ee-b874-88a0e3e77998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:05.730273 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b0a55-9b5e-44ee-b874-88a0e3e77998 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:05.740014 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 71/145] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:05.745983 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc4511-0fec-477b-98ac-ac2eeb852658 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:05.812826 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:05.885195 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc4511-0fec-477b-98ac-ac2eeb852658 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:05.932550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b491472-ce26-4625-a17a-86f4155e918f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:05.933258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b491472-ce26-4625-a17a-86f4155e918f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:05.933534 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b491472-ce26-4625-a17a-86f4155e918f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:05.933852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b491472-ce26-4625-a17a-86f4155e918f None admin] PATH_INFO: `/v3/users/2ffc232160674b439f3975bfd4232051` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:05.935264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b491472-ce26-4625-a17a-86f4155e918f None admin] RBAC: Authorizing `identity:get_user(user_id=2ffc232160674b439f3975bfd4232051)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:05.936808 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b491472-ce26-4625-a17a-86f4155e918f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:05.938601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 75/146] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:05 2022] GET /identity/v3/users/2ffc232160674b439f3975bfd4232051 => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:05.945027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b0a55-9b5e-44ee-b874-88a0e3e77998 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:06.023939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:06.052511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b0a55-9b5e-44ee-b874-88a0e3e77998 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:06.089718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c5b78-a8a2-4038-99d1-4cbd49f00963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:06.090467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c5b78-a8a2-4038-99d1-4cbd49f00963 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:06.090735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c5b78-a8a2-4038-99d1-4cbd49f00963 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:06.091001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c5b78-a8a2-4038-99d1-4cbd49f00963 None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:06.092192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c5b78-a8a2-4038-99d1-4cbd49f00963 None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:06.093708 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c5b78-a8a2-4038-99d1-4cbd49f00963 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:06.095403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 72/147] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:05 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:06.101856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b491472-ce26-4625-a17a-86f4155e918f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:06.272002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b491472-ce26-4625-a17a-86f4155e918f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:06.299497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:06.331994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d426db-e33e-40bc-bdf4-03d4f8c5bc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:06.333055 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d426db-e33e-40bc-bdf4-03d4f8c5bc41 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:06.333547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d426db-e33e-40bc-bdf4-03d4f8c5bc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:06.333998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d426db-e33e-40bc-bdf4-03d4f8c5bc41 None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700/users/2ffc232160674b439f3975bfd4232051/roles/712114868a39430b80ec9c19cb91c947` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:06.336134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d426db-e33e-40bc-bdf4-03d4f8c5bc41 None admin] RBAC: Authorizing `identity:create_grant(project_id=9edb599ff8bb4c67b467b712ea2dc700, user_id=2ffc232160674b439f3975bfd4232051, role_id=712114868a39430b80ec9c19cb91c947)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:06.337907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d426db-e33e-40bc-bdf4-03d4f8c5bc41 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:06.364909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 76/148] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:04:06 2022] PUT /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700/users/2ffc232160674b439f3975bfd4232051/roles/712114868a39430b80ec9c19cb91c947 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:07.827887 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fb78f-bf15-4aa5-a386-30189820a6d3 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:07.828517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fb78f-bf15-4aa5-a386-30189820a6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:07.828517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fb78f-bf15-4aa5-a386-30189820a6d3 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:07.829931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 73/149] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:07.835705 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba65a1-1881-427b-b960-c886770da250 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:07.836622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba65a1-1881-427b-b960-c886770da250 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:07.837104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba65a1-1881-427b-b960-c886770da250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:07.855640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eacb08-2ef4-44f3-8122-410b93a79fb9 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:07.856370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eacb08-2ef4-44f3-8122-410b93a79fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:07.856894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eacb08-2ef4-44f3-8122-410b93a79fb9 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:07.859097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 74/150] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:07.870582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a5ba65a1-1881-427b-b960-c886770da250 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:07.872192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a5ba65a1-1881-427b-b960-c886770da250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:07.874545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b77fc-03aa-4f0c-af6a-14000f4bae65 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:07.875069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b77fc-03aa-4f0c-af6a-14000f4bae65 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:07.875537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b77fc-03aa-4f0c-af6a-14000f4bae65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:07.900618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c63b77fc-03aa-4f0c-af6a-14000f4bae65 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:07.900618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c63b77fc-03aa-4f0c-af6a-14000f4bae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:07.998093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 77/151] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:07.999806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da1d3a-5750-4704-b455-6de6e834c6d2 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:08.000301 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da1d3a-5750-4704-b455-6de6e834c6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:08.000808 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3da1d3a-5750-4704-b455-6de6e834c6d2 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:08.002729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 78/152] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:08.010419 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71064796-690b-4c0f-a709-e39f7a902b3a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:08.011127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71064796-690b-4c0f-a709-e39f7a902b3a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:08.011604 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71064796-690b-4c0f-a709-e39f7a902b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:08.013162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 75/153] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:08.046844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-71064796-690b-4c0f-a709-e39f7a902b3a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:08.048434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-71064796-690b-4c0f-a709-e39f7a902b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:08.164226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 79/154] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:08.228781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2f406-f908-414b-8e3a-4611eb094ec6 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:08.229471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2f406-f908-414b-8e3a-4611eb094ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:08.230033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2f406-f908-414b-8e3a-4611eb094ec6 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:08.231867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 76/155] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:08.246577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb6827-1324-4128-8b88-13e99b972079 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:08.247238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb6827-1324-4128-8b88-13e99b972079 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:08.247716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb6827-1324-4128-8b88-13e99b972079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:08.278321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6deb6827-1324-4128-8b88-13e99b972079 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:08.281015 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6deb6827-1324-4128-8b88-13e99b972079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:08.325773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:08.383151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 80/156] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:08.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2f406-f908-414b-8e3a-4611eb094ec6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:08.590373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:08.628759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2f406-f908-414b-8e3a-4611eb094ec6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:08.670637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00460057-5f79-430f-9aa8-15251cb64b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:08.671264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00460057-5f79-430f-9aa8-15251cb64b00 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:08.671534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00460057-5f79-430f-9aa8-15251cb64b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:08.671797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00460057-5f79-430f-9aa8-15251cb64b00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:08.685524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00460057-5f79-430f-9aa8-15251cb64b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-00460057-5f79-430f-9aa8-15251cb64b00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:08.690524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:08.691399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:08.692170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:08.692170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:08.692170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:08.692170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:08.692170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:08.692170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:08.692170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:08.692170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:08.692170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:08.692170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:08.692170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:08.692170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:08.692170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 77/157] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:04:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:08.699794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb6827-1324-4128-8b88-13e99b972079 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:08.743111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00460057-5f79-430f-9aa8-15251cb64b00 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:08.871430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb6827-1324-4128-8b88-13e99b972079 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:08.945928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316be06e-70ca-409f-900b-9d14cd1293a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:08.946556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316be06e-70ca-409f-900b-9d14cd1293a9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:08.946827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316be06e-70ca-409f-900b-9d14cd1293a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:08.947089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316be06e-70ca-409f-900b-9d14cd1293a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:08.947945 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316be06e-70ca-409f-900b-9d14cd1293a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:08.948977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316be06e-70ca-409f-900b-9d14cd1293a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:08.950401 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316be06e-70ca-409f-900b-9d14cd1293a9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:08.957396 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00460057-5f79-430f-9aa8-15251cb64b00 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:08.962210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 81/158] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:08.962928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316be06e-70ca-409f-900b-9d14cd1293a9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:09.008952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:09.019633 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:09.037302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15d995-a246-4f81-8202-38484e045e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:09.037950 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15d995-a246-4f81-8202-38484e045e1a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:09.038205 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15d995-a246-4f81-8202-38484e045e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:09.038473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15d995-a246-4f81-8202-38484e045e1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:09.065286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15d995-a246-4f81-8202-38484e045e1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-4e15d995-a246-4f81-8202-38484e045e1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:09.079670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:09.080700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:09.081473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:09.081473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:09.081473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:09.081473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:09.081473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:09.081473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:09.081473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:09.081473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:09.081473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:09.081473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:09.081473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:09.081473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:09.081473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 78/159] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:04:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:04:09.082143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15d995-a246-4f81-8202-38484e045e1a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:09.176273 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316be06e-70ca-409f-900b-9d14cd1293a9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:09.269392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60eae64-f860-487b-ad0a-a401c183729c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:09.270053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60eae64-f860-487b-ad0a-a401c183729c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:09.270332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60eae64-f860-487b-ad0a-a401c183729c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:09.270604 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60eae64-f860-487b-ad0a-a401c183729c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:09.277858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15d995-a246-4f81-8202-38484e045e1a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:09.302032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60eae64-f860-487b-ad0a-a401c183729c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-c60eae64-f860-487b-ad0a-a401c183729c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:09.307477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:09.308312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:09.309399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:09.309399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:09.309399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:09.309399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:09.309399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:09.309399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:09.309399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:09.309399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:09.309399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:09.309399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:09.309399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:09.309399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:09.314525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 82/160] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:09.315392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60eae64-f860-487b-ad0a-a401c183729c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:09.350405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff7376-8593-4144-906e-935a89ac6487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:09.351055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff7376-8593-4144-906e-935a89ac6487 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:09.351316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff7376-8593-4144-906e-935a89ac6487 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:09.351579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff7376-8593-4144-906e-935a89ac6487 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:09.381089 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff7376-8593-4144-906e-935a89ac6487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-a8ff7376-8593-4144-906e-935a89ac6487 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:09.386002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:09.386861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:09.387621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:09.387621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:09.387621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:09.387621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:09.387621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:09.387621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:09.387621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:09.387621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:09.387621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:09.387621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:09.387621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:09.387621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:09.388447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 79/161] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:09.391547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff7376-8593-4144-906e-935a89ac6487 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:09.525617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:09.538390 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:09.554676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60eae64-f860-487b-ad0a-a401c183729c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:09.617501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45087e85-5207-4de6-91c6-6ede000b7545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:09.618161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45087e85-5207-4de6-91c6-6ede000b7545 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:09.618441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45087e85-5207-4de6-91c6-6ede000b7545 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:09.618713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45087e85-5207-4de6-91c6-6ede000b7545 None admin] PATH_INFO: `/v3/users/1d564d82faf2482faf5205415837bf68` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:09.630967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45087e85-5207-4de6-91c6-6ede000b7545 None admin] RBAC: Authorizing `identity:get_user(user_id=1d564d82faf2482faf5205415837bf68)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:09.630967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45087e85-5207-4de6-91c6-6ede000b7545 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:09.630967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 83/162] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:09 2022] GET /identity/v3/users/1d564d82faf2482faf5205415837bf68 => generated 247 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:09.630967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45087e85-5207-4de6-91c6-6ede000b7545 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:09.686763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff7376-8593-4144-906e-935a89ac6487 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:09.781446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e710471-d801-427e-b063-2f8d65f6df70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:09.782014 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e710471-d801-427e-b063-2f8d65f6df70 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:09.782311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e710471-d801-427e-b063-2f8d65f6df70 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:09.782585 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e710471-d801-427e-b063-2f8d65f6df70 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:09.783270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e710471-d801-427e-b063-2f8d65f6df70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:09.783564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e710471-d801-427e-b063-2f8d65f6df70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:09.784994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e710471-d801-427e-b063-2f8d65f6df70 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:09.786745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45087e85-5207-4de6-91c6-6ede000b7545 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:09.803886 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 80/163] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:04:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:09.804776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e710471-d801-427e-b063-2f8d65f6df70 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:09.827742 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dd02b-7e68-4960-8f0b-24cbfd7d7a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:09.828743 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dd02b-7e68-4960-8f0b-24cbfd7d7a78 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:09.829232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dd02b-7e68-4960-8f0b-24cbfd7d7a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:09.829686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4dd02b-7e68-4960-8f0b-24cbfd7d7a78 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:09.831044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dd02b-7e68-4960-8f0b-24cbfd7d7a78 None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:09.831541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dd02b-7e68-4960-8f0b-24cbfd7d7a78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:09.833215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4dd02b-7e68-4960-8f0b-24cbfd7d7a78 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:09.843587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 84/164] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:09.844920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dd02b-7e68-4960-8f0b-24cbfd7d7a78 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:09.872700 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:10.013248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4dd02b-7e68-4960-8f0b-24cbfd7d7a78 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:10.043706 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e710471-d801-427e-b063-2f8d65f6df70 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:10.076850 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496e062-12a6-4508-afc1-556db908b84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:10.077079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:10.078020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496e062-12a6-4508-afc1-556db908b84c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:10.078575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496e062-12a6-4508-afc1-556db908b84c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:10.079036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496e062-12a6-4508-afc1-556db908b84c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:10.093333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95787ee-3661-44e3-b3eb-0ee17e8945af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:10.093954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95787ee-3661-44e3-b3eb-0ee17e8945af None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:10.094489 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95787ee-3661-44e3-b3eb-0ee17e8945af None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:10.094489 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95787ee-3661-44e3-b3eb-0ee17e8945af None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:10.095148 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95787ee-3661-44e3-b3eb-0ee17e8945af None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:10.095442 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95787ee-3661-44e3-b3eb-0ee17e8945af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:10.096847 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95787ee-3661-44e3-b3eb-0ee17e8945af None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:10.122706 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 81/165] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:10.123593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95787ee-3661-44e3-b3eb-0ee17e8945af None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:10.169371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496e062-12a6-4508-afc1-556db908b84c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:10.171718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496e062-12a6-4508-afc1-556db908b84c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:10.205058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:10.259886 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95787ee-3661-44e3-b3eb-0ee17e8945af None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:10.304204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242be1f-46e8-421f-b0e0-0f45f1026f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:10.304780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242be1f-46e8-421f-b0e0-0f45f1026f90 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:10.305050 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242be1f-46e8-421f-b0e0-0f45f1026f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:10.305310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242be1f-46e8-421f-b0e0-0f45f1026f90 None admin] PATH_INFO: `/v3/users/4cdec6738f534b16af19e42f8f462d49` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:10.306531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242be1f-46e8-421f-b0e0-0f45f1026f90 None admin] RBAC: Authorizing `identity:get_user(user_id=4cdec6738f534b16af19e42f8f462d49)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:10.308057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242be1f-46e8-421f-b0e0-0f45f1026f90 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:10.309707 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 82/166] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:10 2022] GET /identity/v3/users/4cdec6738f534b16af19e42f8f462d49 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:10.310469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242be1f-46e8-421f-b0e0-0f45f1026f90 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:10.377828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 85/167] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:10.378505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496e062-12a6-4508-afc1-556db908b84c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:10.383018 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:10.505630 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242be1f-46e8-421f-b0e0-0f45f1026f90 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:10.551748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4d8f6-d4da-408a-aaba-bd0b8dfccb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:10.551893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496e062-12a6-4508-afc1-556db908b84c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:10.551992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4d8f6-d4da-408a-aaba-bd0b8dfccb55 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:10.551992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4d8f6-d4da-408a-aaba-bd0b8dfccb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:10.551992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4d8f6-d4da-408a-aaba-bd0b8dfccb55 None admin] PATH_INFO: `/v3/users/4cdec6738f534b16af19e42f8f462d49` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:10.552461 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4d8f6-d4da-408a-aaba-bd0b8dfccb55 None admin] RBAC: Authorizing `identity:get_user(user_id=4cdec6738f534b16af19e42f8f462d49)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:10.554037 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4d8f6-d4da-408a-aaba-bd0b8dfccb55 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:10.555931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 83/168] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:10 2022] GET /identity/v3/users/4cdec6738f534b16af19e42f8f462d49 => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:10.556820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05e175-ed78-46e3-b9bd-00eaa5a6969c None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:10.557122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05e175-ed78-46e3-b9bd-00eaa5a6969c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:10.557386 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05e175-ed78-46e3-b9bd-00eaa5a6969c None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:10.558576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 84/169] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:10.559469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05e175-ed78-46e3-b9bd-00eaa5a6969c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:10.582309 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:10.625948 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b5935-88ac-40ce-9851-bc5155c307ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:10.626646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b5935-88ac-40ce-9851-bc5155c307ca None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:10.626920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b5935-88ac-40ce-9851-bc5155c307ca None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:10.627183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b5935-88ac-40ce-9851-bc5155c307ca None admin] PATH_INFO: `/v3/users/4cdec6738f534b16af19e42f8f462d49` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:10.633541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b5935-88ac-40ce-9851-bc5155c307ca None admin] RBAC: Authorizing `identity:get_user(user_id=4cdec6738f534b16af19e42f8f462d49)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:10.635035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b5935-88ac-40ce-9851-bc5155c307ca None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:10.637227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 86/170] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:10 2022] GET /identity/v3/users/4cdec6738f534b16af19e42f8f462d49 => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:10.637780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b5935-88ac-40ce-9851-bc5155c307ca None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:10.683689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05e175-ed78-46e3-b9bd-00eaa5a6969c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:10.698312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:10.721172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f93cb-f11f-42ed-9224-9116f6b78b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:10.721833 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f93cb-f11f-42ed-9224-9116f6b78b6f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:10.722105 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f93cb-f11f-42ed-9224-9116f6b78b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:10.722369 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f93cb-f11f-42ed-9224-9116f6b78b6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:10.758389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b5935-88ac-40ce-9851-bc5155c307ca None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:10.794718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df56b22-d69f-48d4-92c8-9f6749ef129e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:10.795350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df56b22-d69f-48d4-92c8-9f6749ef129e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:10.795619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df56b22-d69f-48d4-92c8-9f6749ef129e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:10.795877 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df56b22-d69f-48d4-92c8-9f6749ef129e None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:10.797034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df56b22-d69f-48d4-92c8-9f6749ef129e None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:10.798503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df56b22-d69f-48d4-92c8-9f6749ef129e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:10.799797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f93cb-f11f-42ed-9224-9116f6b78b6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:10.800141 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 87/171] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:10 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:10.801608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f93cb-f11f-42ed-9224-9116f6b78b6f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:10.805740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df56b22-d69f-48d4-92c8-9f6749ef129e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:10.877044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:10.909600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 85/172] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:10.911020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6aebd-731b-45ba-8e77-3e591d2fae3f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:10.911388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6aebd-731b-45ba-8e77-3e591d2fae3f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:10.911721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6aebd-731b-45ba-8e77-3e591d2fae3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:10.932552 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-23f6aebd-731b-45ba-8e77-3e591d2fae3f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:10.933876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-23f6aebd-731b-45ba-8e77-3e591d2fae3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:10.934083 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df56b22-d69f-48d4-92c8-9f6749ef129e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:10.944630 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:10.976554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e9499-ad4b-4ca4-8ee0-bf7639aa7085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:10.977190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e9499-ad4b-4ca4-8ee0-bf7639aa7085 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:10.978143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e9499-ad4b-4ca4-8ee0-bf7639aa7085 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:10.978143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e9499-ad4b-4ca4-8ee0-bf7639aa7085 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:11.012231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 86/173] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:11.013219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6aebd-731b-45ba-8e77-3e591d2fae3f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:11.076711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e9499-ad4b-4ca4-8ee0-bf7639aa7085 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:11.078188 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e9499-ad4b-4ca4-8ee0-bf7639aa7085 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:11.130076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6aebd-731b-45ba-8e77-3e591d2fae3f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:11.133685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:11.169416 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e671de-a24f-4a97-9a86-e7375237dafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:11.170055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e671de-a24f-4a97-9a86-e7375237dafe None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:11.170326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e671de-a24f-4a97-9a86-e7375237dafe None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:11.170589 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e671de-a24f-4a97-9a86-e7375237dafe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:11.184430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:11.190104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 88/174] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:11.191434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33e9c7-c5bb-4e71-a8fa-845fe67b95c7 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:11.191724 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33e9c7-c5bb-4e71-a8fa-845fe67b95c7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:11.192141 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33e9c7-c5bb-4e71-a8fa-845fe67b95c7 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:11.193914 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 89/175] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:11.197388 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02b8ab-7782-47df-8ffa-338b33800a7f None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:11.197849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02b8ab-7782-47df-8ffa-338b33800a7f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:11.198940 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02b8ab-7782-47df-8ffa-338b33800a7f None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:11.200315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 90/176] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:11.201261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02b8ab-7782-47df-8ffa-338b33800a7f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:11.254523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e671de-a24f-4a97-9a86-e7375237dafe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:11.256001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e671de-a24f-4a97-9a86-e7375237dafe None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:11.313929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02b8ab-7782-47df-8ffa-338b33800a7f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:11.348177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d038d6f-28cf-4110-a137-29ee19b0088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:11.348813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d038d6f-28cf-4110-a137-29ee19b0088b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:11.349101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d038d6f-28cf-4110-a137-29ee19b0088b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:11.349344 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d038d6f-28cf-4110-a137-29ee19b0088b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:11.350069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d038d6f-28cf-4110-a137-29ee19b0088b None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=9edb599ff8bb4c67b467b712ea2dc700, user.id=1d564d82faf2482faf5205415837bf68) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:11.350368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d038d6f-28cf-4110-a137-29ee19b0088b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:11.351803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d038d6f-28cf-4110-a137-29ee19b0088b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:11.363687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 91/177] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:11 2022] GET /identity/v3/role_assignments?user.id=1d564d82faf2482faf5205415837bf68&scope.project.id=9edb599ff8bb4c67b467b712ea2dc700&role.id=ae444a489623434999d758b71ae2dc87 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:11.363687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d038d6f-28cf-4110-a137-29ee19b0088b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:11.369056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 87/178] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:11.370712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442dab3-dad5-4466-9fbb-6f10b9dcfae9 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:11.371067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442dab3-dad5-4466-9fbb-6f10b9dcfae9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:11.371343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442dab3-dad5-4466-9fbb-6f10b9dcfae9 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:11.372960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 88/179] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:11.373622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442dab3-dad5-4466-9fbb-6f10b9dcfae9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:11.391337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:11.442941 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:11.488060 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d038d6f-28cf-4110-a137-29ee19b0088b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:11.517671 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442dab3-dad5-4466-9fbb-6f10b9dcfae9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:11.545527 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee5577-59a2-44d0-8f37-a1a8621ad47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:11.546411 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee5577-59a2-44d0-8f37-a1a8621ad47b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:11.546684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee5577-59a2-44d0-8f37-a1a8621ad47b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:11.546939 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee5577-59a2-44d0-8f37-a1a8621ad47b None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:11.549596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee5577-59a2-44d0-8f37-a1a8621ad47b None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:11.551097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee5577-59a2-44d0-8f37-a1a8621ad47b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:11.552991 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 92/180] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:11 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:11.554123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc41061-9219-4402-9226-7e620b9f5b1d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:11.554389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc41061-9219-4402-9226-7e620b9f5b1d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:11.554655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc41061-9219-4402-9226-7e620b9f5b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:11.572218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270e759-28fa-4d86-b750-fa0349902dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:11.574336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270e759-28fa-4d86-b750-fa0349902dad None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:11.574642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270e759-28fa-4d86-b750-fa0349902dad None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:11.575742 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270e759-28fa-4d86-b750-fa0349902dad None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:11.577595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270e759-28fa-4d86-b750-fa0349902dad None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:11.579542 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270e759-28fa-4d86-b750-fa0349902dad None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:11.582041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 89/181] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:11 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:11.584110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d350da-0f13-4423-a556-650f594016ce None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:11.584569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d350da-0f13-4423-a556-650f594016ce None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:11.585071 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d350da-0f13-4423-a556-650f594016ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:11.587233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-dbc41061-9219-4402-9226-7e620b9f5b1d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:11.588766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dbc41061-9219-4402-9226-7e620b9f5b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:11.611082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c6d350da-0f13-4423-a556-650f594016ce None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:11.612793 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c6d350da-0f13-4423-a556-650f594016ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:11.671333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 93/182] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:11.672089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc41061-9219-4402-9226-7e620b9f5b1d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:11.687835 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:11.697122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 90/183] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:11.698266 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d350da-0f13-4423-a556-650f594016ce None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:11.750319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:11.810258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc41061-9219-4402-9226-7e620b9f5b1d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:11.821140 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d350da-0f13-4423-a556-650f594016ce None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:11.864746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef232881-4357-4579-9a85-0c3cdc180eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:11.865168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bc272-3694-429d-88d2-4ffbb2b83570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:11.865404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef232881-4357-4579-9a85-0c3cdc180eaa None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:11.865677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef232881-4357-4579-9a85-0c3cdc180eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:11.865835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bc272-3694-429d-88d2-4ffbb2b83570 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:11.865934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef232881-4357-4579-9a85-0c3cdc180eaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:11.866096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bc272-3694-429d-88d2-4ffbb2b83570 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:11.866603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bc272-3694-429d-88d2-4ffbb2b83570 None admin] PATH_INFO: `/v3/projects/84e33566ed254f9fb0adf234a6d5ec15` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:11.872087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bc272-3694-429d-88d2-4ffbb2b83570 None admin] RBAC: Authorizing `identity:get_project(project_id=84e33566ed254f9fb0adf234a6d5ec15)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:11.872087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bc272-3694-429d-88d2-4ffbb2b83570 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:11.872087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 91/184] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:11 2022] GET /identity/v3/projects/84e33566ed254f9fb0adf234a6d5ec15 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:11.873178 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906616c-34ad-4b4e-ae31-c10cf9a3e882 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:11.873453 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906616c-34ad-4b4e-ae31-c10cf9a3e882 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:11.873721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906616c-34ad-4b4e-ae31-c10cf9a3e882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:11.890408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef232881-4357-4579-9a85-0c3cdc180eaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-ef232881-4357-4579-9a85-0c3cdc180eaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:11.896138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:11.897062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:11.897852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:11.897852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:11.897852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:11.897852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:11.897852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:11.897852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:11.897852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:11.897852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:11.897852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:11.897852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:11.897852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:11.897852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:11.899922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 94/185] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:04:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:11.900931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef232881-4357-4579-9a85-0c3cdc180eaa None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:11.901071 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b906616c-34ad-4b4e-ae31-c10cf9a3e882 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:11.902395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b906616c-34ad-4b4e-ae31-c10cf9a3e882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:11.976512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 92/186] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:11.977512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906616c-34ad-4b4e-ae31-c10cf9a3e882 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:11.996928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:12.019799 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:12.031938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef232881-4357-4579-9a85-0c3cdc180eaa None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:12.071504 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56df765-0857-48ec-8151-9fe37863cc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:12.072441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df765-0857-48ec-8151-9fe37863cc24 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:12.072441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df765-0857-48ec-8151-9fe37863cc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:12.072749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56df765-0857-48ec-8151-9fe37863cc24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:12.073579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56df765-0857-48ec-8151-9fe37863cc24 None admin] RBAC: Adding query filter params (role.id=704c67b455b54fb48089faef4aba8ccd, scope.project.id=9edb599ff8bb4c67b467b712ea2dc700, user.id=4cdec6738f534b16af19e42f8f462d49) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:12.073918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56df765-0857-48ec-8151-9fe37863cc24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:12.075575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56df765-0857-48ec-8151-9fe37863cc24 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:12.082275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 95/187] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:11 2022] GET /identity/v3/role_assignments?user.id=4cdec6738f534b16af19e42f8f462d49&scope.project.id=9edb599ff8bb4c67b467b712ea2dc700&role.id=704c67b455b54fb48089faef4aba8ccd => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:12.082992 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56df765-0857-48ec-8151-9fe37863cc24 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:12.091242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906616c-34ad-4b4e-ae31-c10cf9a3e882 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:12.151141 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d0444-a6fb-431b-95e4-a699f86a6db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:12.153352 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d0444-a6fb-431b-95e4-a699f86a6db1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:12.154087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d0444-a6fb-431b-95e4-a699f86a6db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:12.154551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d0444-a6fb-431b-95e4-a699f86a6db1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:12.156195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d0444-a6fb-431b-95e4-a699f86a6db1 None admin] RBAC: Adding query filter params (role.id=2921a7146fd9408396ca98a4be4ecd3f, scope.project.id=9edb599ff8bb4c67b467b712ea2dc700, user.id=4cdec6738f534b16af19e42f8f462d49) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:12.157260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d0444-a6fb-431b-95e4-a699f86a6db1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:12.159457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d0444-a6fb-431b-95e4-a699f86a6db1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:12.174290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 93/188] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:11 2022] GET /identity/v3/role_assignments?user.id=4cdec6738f534b16af19e42f8f462d49&scope.project.id=9edb599ff8bb4c67b467b712ea2dc700&role.id=2921a7146fd9408396ca98a4be4ecd3f => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:12.174290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d0444-a6fb-431b-95e4-a699f86a6db1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:12.254096 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56df765-0857-48ec-8151-9fe37863cc24 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:12.305367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415a094-0303-46be-9903-fd286bc3b255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:12.307389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415a094-0303-46be-9903-fd286bc3b255 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:12.307574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415a094-0303-46be-9903-fd286bc3b255 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:12.307871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415a094-0303-46be-9903-fd286bc3b255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:12.309386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415a094-0303-46be-9903-fd286bc3b255 None admin] RBAC: Adding query filter params (role.id=2921a7146fd9408396ca98a4be4ecd3f, scope.project.id=84e33566ed254f9fb0adf234a6d5ec15, user.id=4cdec6738f534b16af19e42f8f462d49) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:12.310039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415a094-0303-46be-9903-fd286bc3b255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:12.312652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415a094-0303-46be-9903-fd286bc3b255 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:12.321573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 96/189] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:12 2022] GET /identity/v3/role_assignments?user.id=4cdec6738f534b16af19e42f8f462d49&scope.project.id=84e33566ed254f9fb0adf234a6d5ec15&role.id=2921a7146fd9408396ca98a4be4ecd3f => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:12.322429 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415a094-0303-46be-9903-fd286bc3b255 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:12.325849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:12.345968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d0444-a6fb-431b-95e4-a699f86a6db1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:12.363446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:12.393000 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c36eb-1434-4539-963e-645b39c717ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:12.393678 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c36eb-1434-4539-963e-645b39c717ed None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:12.393947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c36eb-1434-4539-963e-645b39c717ed None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:12.394215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c36eb-1434-4539-963e-645b39c717ed None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:12.394912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c36eb-1434-4539-963e-645b39c717ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:12.395205 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c36eb-1434-4539-963e-645b39c717ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:12.396637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c36eb-1434-4539-963e-645b39c717ed None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:12.408882 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 94/190] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:12.409797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c36eb-1434-4539-963e-645b39c717ed None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:12.460041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415a094-0303-46be-9903-fd286bc3b255 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:12.503487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d5d45-60b1-43ca-b3e6-2706f75d5b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:12.504202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d5d45-60b1-43ca-b3e6-2706f75d5b14 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:12.504400 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d5d45-60b1-43ca-b3e6-2706f75d5b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:12.504717 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d5d45-60b1-43ca-b3e6-2706f75d5b14 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:12.522213 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d5d45-60b1-43ca-b3e6-2706f75d5b14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-de8d5d45-60b1-43ca-b3e6-2706f75d5b14 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:12.529779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:12.530735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:12.531509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:12.531509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:12.531509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:12.531509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:12.531509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:12.531509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:12.531509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:12.531509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:12.531509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:12.531509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:12.531509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:12.531509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:12.531509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 97/191] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:12.532164 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d5d45-60b1-43ca-b3e6-2706f75d5b14 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:12.553087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c36eb-1434-4539-963e-645b39c717ed None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:12.594572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69d395-d701-46ab-ad9c-36ea6c9263f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:12.595210 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69d395-d701-46ab-ad9c-36ea6c9263f7 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:12.595479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69d395-d701-46ab-ad9c-36ea6c9263f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:12.595743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69d395-d701-46ab-ad9c-36ea6c9263f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:12.606652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69d395-d701-46ab-ad9c-36ea6c9263f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-0e69d395-d701-46ab-ad9c-36ea6c9263f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:12.612327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:12.613349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:12.614123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:12.614123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:12.614123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:12.614123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:12.614123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:12.614123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:12.614123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:12.614123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:12.614123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:12.614123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:12.614123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:12.614123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:12.614123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 95/192] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:04:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:12.614752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69d395-d701-46ab-ad9c-36ea6c9263f7 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:12.629852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:12.666066 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:12.702363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d5d45-60b1-43ca-b3e6-2706f75d5b14 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:12.749679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69d395-d701-46ab-ad9c-36ea6c9263f7 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:12.759703 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7a483-33d0-41eb-aa29-08262a06ef4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:12.760952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7a483-33d0-41eb-aa29-08262a06ef4b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:12.761148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7a483-33d0-41eb-aa29-08262a06ef4b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:12.761297 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7a483-33d0-41eb-aa29-08262a06ef4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:12.762549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7a483-33d0-41eb-aa29-08262a06ef4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:12.764011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7a483-33d0-41eb-aa29-08262a06ef4b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:12.765690 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 98/193] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:12.771624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7a483-33d0-41eb-aa29-08262a06ef4b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:12.805213 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf7c6b-b646-4b97-a002-ea9bd0908326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:12.806354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf7c6b-b646-4b97-a002-ea9bd0908326 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:12.806803 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf7c6b-b646-4b97-a002-ea9bd0908326 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:12.807273 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf7c6b-b646-4b97-a002-ea9bd0908326 None admin] PATH_INFO: `/v3/users/20a038652c604d3db04c261857dd4e93` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:12.840638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf7c6b-b646-4b97-a002-ea9bd0908326 None admin] RBAC: Authorizing `identity:get_user(user_id=20a038652c604d3db04c261857dd4e93)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:12.840638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf7c6b-b646-4b97-a002-ea9bd0908326 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:12.841813 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 96/194] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:12 2022] GET /identity/v3/users/20a038652c604d3db04c261857dd4e93 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:12.842630 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf7c6b-b646-4b97-a002-ea9bd0908326 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:12.914922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7a483-33d0-41eb-aa29-08262a06ef4b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:12.932412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:12.947562 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:12.952972 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf7c6b-b646-4b97-a002-ea9bd0908326 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:12.954788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a992a-5bc4-4227-8be8-5c42ccde9a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:12.955409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a992a-5bc4-4227-8be8-5c42ccde9a6d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:12.955680 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a992a-5bc4-4227-8be8-5c42ccde9a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:12.955938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a992a-5bc4-4227-8be8-5c42ccde9a6d None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:12.956642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a992a-5bc4-4227-8be8-5c42ccde9a6d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:12.956943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a992a-5bc4-4227-8be8-5c42ccde9a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:12.958316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a992a-5bc4-4227-8be8-5c42ccde9a6d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:12.968725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 99/195] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:12.969466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a992a-5bc4-4227-8be8-5c42ccde9a6d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:12.996116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846c96a-f7cf-4f9d-ba0c-4d32cb414b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:12.996782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846c96a-f7cf-4f9d-ba0c-4d32cb414b34 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:12.997054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846c96a-f7cf-4f9d-ba0c-4d32cb414b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:12.997315 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846c96a-f7cf-4f9d-ba0c-4d32cb414b34 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:12.997992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846c96a-f7cf-4f9d-ba0c-4d32cb414b34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:12.998290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846c96a-f7cf-4f9d-ba0c-4d32cb414b34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:12.999677 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846c96a-f7cf-4f9d-ba0c-4d32cb414b34 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:13.008689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 97/196] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:04:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:13.013762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846c96a-f7cf-4f9d-ba0c-4d32cb414b34 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:13.074863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a992a-5bc4-4227-8be8-5c42ccde9a6d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:13.111136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72d193-2a39-45ef-8890-546f649c7a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:13.111747 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72d193-2a39-45ef-8890-546f649c7a6d None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:13.112013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72d193-2a39-45ef-8890-546f649c7a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:13.112278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72d193-2a39-45ef-8890-546f649c7a6d None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:13.112973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72d193-2a39-45ef-8890-546f649c7a6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:13.114436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72d193-2a39-45ef-8890-546f649c7a6d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:13.118250 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846c96a-f7cf-4f9d-ba0c-4d32cb414b34 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:13.143738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 100/197] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:04:12 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:04:13.144712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72d193-2a39-45ef-8890-546f649c7a6d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:13.155546 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a943054-72ee-4ca4-ba8a-cd0a82e6f30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:13.156261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a943054-72ee-4ca4-ba8a-cd0a82e6f30e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:13.156522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a943054-72ee-4ca4-ba8a-cd0a82e6f30e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:13.156856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a943054-72ee-4ca4-ba8a-cd0a82e6f30e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:13.175096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:13.206337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:13.252254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a943054-72ee-4ca4-ba8a-cd0a82e6f30e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:13.254366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a943054-72ee-4ca4-ba8a-cd0a82e6f30e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:13.270216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72d193-2a39-45ef-8890-546f649c7a6d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:13.306841 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9389a-c09b-40bf-843e-36c042f1e5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:13.307710 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9389a-c09b-40bf-843e-36c042f1e5ee None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:13.307896 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9389a-c09b-40bf-843e-36c042f1e5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:13.308172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9389a-c09b-40bf-843e-36c042f1e5ee None admin] PATH_INFO: `/v3/users/2ffc232160674b439f3975bfd4232051` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:13.310021 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9389a-c09b-40bf-843e-36c042f1e5ee None admin] RBAC: Authorizing `identity:get_user(user_id=2ffc232160674b439f3975bfd4232051)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:13.311525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9389a-c09b-40bf-843e-36c042f1e5ee None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:13.313189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 101/198] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:13 2022] GET /identity/v3/users/2ffc232160674b439f3975bfd4232051 => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:13.314071 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9389a-c09b-40bf-843e-36c042f1e5ee None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:13.376842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 98/199] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:13.377336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a943054-72ee-4ca4-ba8a-cd0a82e6f30e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:13.434479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:13.440511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9389a-c09b-40bf-843e-36c042f1e5ee None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:13.460391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:13.490088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c1e73-571c-4c2e-870f-9df19a14992f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:13.491011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c1e73-571c-4c2e-870f-9df19a14992f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:13.491335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c1e73-571c-4c2e-870f-9df19a14992f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:13.491602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c1e73-571c-4c2e-870f-9df19a14992f None admin] PATH_INFO: `/v3/users/20a038652c604d3db04c261857dd4e93` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:13.492770 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c1e73-571c-4c2e-870f-9df19a14992f None admin] RBAC: Authorizing `identity:get_user(user_id=20a038652c604d3db04c261857dd4e93)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:13.494225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c1e73-571c-4c2e-870f-9df19a14992f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:13.495881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 102/200] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:13 2022] GET /identity/v3/users/20a038652c604d3db04c261857dd4e93 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:13.496711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c1e73-571c-4c2e-870f-9df19a14992f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:13.512749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a943054-72ee-4ca4-ba8a-cd0a82e6f30e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:13.563281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fe1fd-1498-4091-bbb0-7ae352c8c092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:13.564042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fe1fd-1498-4091-bbb0-7ae352c8c092 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:13.564373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fe1fd-1498-4091-bbb0-7ae352c8c092 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:13.565649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fe1fd-1498-4091-bbb0-7ae352c8c092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:13.635439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c1e73-571c-4c2e-870f-9df19a14992f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:13.666404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fe1fd-1498-4091-bbb0-7ae352c8c092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:13.668171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fe1fd-1498-4091-bbb0-7ae352c8c092 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:13.688407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389ca2b-a1a7-4f05-9721-fb058b1fce23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:13.689208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389ca2b-a1a7-4f05-9721-fb058b1fce23 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:13.689957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389ca2b-a1a7-4f05-9721-fb058b1fce23 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:13.690292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389ca2b-a1a7-4f05-9721-fb058b1fce23 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:13.691568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389ca2b-a1a7-4f05-9721-fb058b1fce23 None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:13.693797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389ca2b-a1a7-4f05-9721-fb058b1fce23 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:13.695941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 103/201] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:13 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:13.696839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389ca2b-a1a7-4f05-9721-fb058b1fce23 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:13.740302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:13.772392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:13.827649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 99/202] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:13.832616 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fe1fd-1498-4091-bbb0-7ae352c8c092 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:13.846646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389ca2b-a1a7-4f05-9721-fb058b1fce23 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:13.899394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9e41b-a668-4b6b-bbb0-e9064b6f6d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:13.900420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9e41b-a668-4b6b-bbb0-e9064b6f6d48 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:13.901024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9e41b-a668-4b6b-bbb0-e9064b6f6d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:13.901731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9e41b-a668-4b6b-bbb0-e9064b6f6d48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:13.965574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fe1fd-1498-4091-bbb0-7ae352c8c092 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:13.988145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9e41b-a668-4b6b-bbb0-e9064b6f6d48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:13.989819 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9e41b-a668-4b6b-bbb0-e9064b6f6d48 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:14.020692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f824b6d-4ca2-4bc6-946f-d322a461bd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:14.021839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f824b6d-4ca2-4bc6-946f-d322a461bd2b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:14.022521 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f824b6d-4ca2-4bc6-946f-d322a461bd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:14.022987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f824b6d-4ca2-4bc6-946f-d322a461bd2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:14.024481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f824b6d-4ca2-4bc6-946f-d322a461bd2b None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=117d6fb504534ef2bd643528f241dffa, user.id=20a038652c604d3db04c261857dd4e93) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:14.024992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f824b6d-4ca2-4bc6-946f-d322a461bd2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:14.028352 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f824b6d-4ca2-4bc6-946f-d322a461bd2b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:14.035714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 100/203] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:13 2022] GET /identity/v3/role_assignments?user.id=20a038652c604d3db04c261857dd4e93&scope.project.id=117d6fb504534ef2bd643528f241dffa&role.id=ae444a489623434999d758b71ae2dc87 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:14.036864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f824b6d-4ca2-4bc6-946f-d322a461bd2b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:14.051923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:14.061058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:14.178638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 104/204] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:14.181704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f824b6d-4ca2-4bc6-946f-d322a461bd2b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:14.184851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9e41b-a668-4b6b-bbb0-e9064b6f6d48 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:14.226697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a368c-0ca7-414c-80ff-9f67874030cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:14.228055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a368c-0ca7-414c-80ff-9f67874030cb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:14.228531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a368c-0ca7-414c-80ff-9f67874030cb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:14.229134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a368c-0ca7-414c-80ff-9f67874030cb None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:14.230912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a368c-0ca7-414c-80ff-9f67874030cb None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:14.233764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a368c-0ca7-414c-80ff-9f67874030cb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:14.236125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 101/205] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:14 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:14.243414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a368c-0ca7-414c-80ff-9f67874030cb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:14.350589 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:14.380526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a368c-0ca7-414c-80ff-9f67874030cb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:14.413544 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9e41b-a668-4b6b-bbb0-e9064b6f6d48 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:14.431522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b1acd-8622-4a0a-93d9-f777dabd4d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:14.432112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b1acd-8622-4a0a-93d9-f777dabd4d40 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:14.432386 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b1acd-8622-4a0a-93d9-f777dabd4d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:14.432712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b1acd-8622-4a0a-93d9-f777dabd4d40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:14.433442 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b1acd-8622-4a0a-93d9-f777dabd4d40 None admin] RBAC: Adding query filter params (role.id=712114868a39430b80ec9c19cb91c947, scope.project.id=9edb599ff8bb4c67b467b712ea2dc700, user.id=2ffc232160674b439f3975bfd4232051) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:14.433741 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b1acd-8622-4a0a-93d9-f777dabd4d40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:14.435176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b1acd-8622-4a0a-93d9-f777dabd4d40 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:14.446455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 102/206] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:14 2022] GET /identity/v3/role_assignments?user.id=2ffc232160674b439f3975bfd4232051&scope.project.id=9edb599ff8bb4c67b467b712ea2dc700&role.id=712114868a39430b80ec9c19cb91c947 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:14.474549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824c925-cbaf-4951-a054-909a81dc4b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:14.475629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824c925-cbaf-4951-a054-909a81dc4b7b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:14.476112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824c925-cbaf-4951-a054-909a81dc4b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:14.476567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824c925-cbaf-4951-a054-909a81dc4b7b None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:14.477927 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824c925-cbaf-4951-a054-909a81dc4b7b None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:14.480313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824c925-cbaf-4951-a054-909a81dc4b7b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:14.482555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 105/207] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:14 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:14.489487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b1acd-8622-4a0a-93d9-f777dabd4d40 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:14.675982 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b1acd-8622-4a0a-93d9-f777dabd4d40 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:14.719884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dca77c-7160-4ee8-be2a-ff5a2f54c9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:14.720408 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dca77c-7160-4ee8-be2a-ff5a2f54c9ee None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:14.720724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dca77c-7160-4ee8-be2a-ff5a2f54c9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:14.720994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dca77c-7160-4ee8-be2a-ff5a2f54c9ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:14.721729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dca77c-7160-4ee8-be2a-ff5a2f54c9ee None admin] RBAC: Adding query filter params (role.id=704c67b455b54fb48089faef4aba8ccd, scope.project.id=117d6fb504534ef2bd643528f241dffa, user.id=20a038652c604d3db04c261857dd4e93) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:14.722034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dca77c-7160-4ee8-be2a-ff5a2f54c9ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:14.723470 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dca77c-7160-4ee8-be2a-ff5a2f54c9ee None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:14.730402 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 103/208] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:14 2022] GET /identity/v3/role_assignments?user.id=20a038652c604d3db04c261857dd4e93&scope.project.id=117d6fb504534ef2bd643528f241dffa&role.id=704c67b455b54fb48089faef4aba8ccd => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:15.811032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63f945-b1c0-404d-987a-e3a133235c0e None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:15.811557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63f945-b1c0-404d-987a-e3a133235c0e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:15.811557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63f945-b1c0-404d-987a-e3a133235c0e None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:15.813301 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 106/209] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:15.819448 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342daea-e38c-4ffe-9567-ecae9ef26699 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:15.819971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342daea-e38c-4ffe-9567-ecae9ef26699 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:15.819971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342daea-e38c-4ffe-9567-ecae9ef26699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:15.827976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:15.840443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8342daea-e38c-4ffe-9567-ecae9ef26699 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:15.841670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8342daea-e38c-4ffe-9567-ecae9ef26699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:15.920726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 104/210] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:15.928772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a580860-3870-486e-9ad6-c691d2b9a633 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:15.929052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a580860-3870-486e-9ad6-c691d2b9a633 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:15.929318 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a580860-3870-486e-9ad6-c691d2b9a633 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:15.930654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 107/211] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:15.936434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8ea1e-2198-48b8-8157-834c52bfbea5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:15.936743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8ea1e-2198-48b8-8157-834c52bfbea5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:15.937008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8ea1e-2198-48b8-8157-834c52bfbea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:15.956412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-64d8ea1e-2198-48b8-8157-834c52bfbea5 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:15.957625 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-64d8ea1e-2198-48b8-8157-834c52bfbea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:16.025998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 105/212] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:16.204366 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a580860-3870-486e-9ad6-c691d2b9a633 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:16.227727 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:16.307455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8ea1e-2198-48b8-8157-834c52bfbea5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:16.319465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a580860-3870-486e-9ad6-c691d2b9a633 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:16.364936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99aa2f-074c-41f8-8651-3300babebad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:16.365593 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99aa2f-074c-41f8-8651-3300babebad3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:16.365863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99aa2f-074c-41f8-8651-3300babebad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:16.366128 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99aa2f-074c-41f8-8651-3300babebad3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:16.381450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99aa2f-074c-41f8-8651-3300babebad3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:16.385067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-8e99aa2f-074c-41f8-8651-3300babebad3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:16.389118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:16.390014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:16.390799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:16.390799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:16.390799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:16.390799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:16.390799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:16.390799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:16.390799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:16.390799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:16.390799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:16.390799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:16.390799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:16.390799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:16.395862 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 108/213] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:16.396797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99aa2f-074c-41f8-8651-3300babebad3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:16.438641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8ea1e-2198-48b8-8157-834c52bfbea5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:16.491405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65941e68-f494-469c-b944-4a6e7f425850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:16.491979 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65941e68-f494-469c-b944-4a6e7f425850 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:16.492253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65941e68-f494-469c-b944-4a6e7f425850 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:16.492520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65941e68-f494-469c-b944-4a6e7f425850 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:16.494310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65941e68-f494-469c-b944-4a6e7f425850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:16.495955 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65941e68-f494-469c-b944-4a6e7f425850 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:16.502991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 106/214] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:16.503932 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65941e68-f494-469c-b944-4a6e7f425850 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:16.505368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:16.528860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99aa2f-074c-41f8-8651-3300babebad3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:16.580795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acda973-4a83-4b1e-8370-4264650a6ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:16.581385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acda973-4a83-4b1e-8370-4264650a6ecb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:16.581660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acda973-4a83-4b1e-8370-4264650a6ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:16.582068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acda973-4a83-4b1e-8370-4264650a6ecb None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:16.582648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acda973-4a83-4b1e-8370-4264650a6ecb None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:16.582947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acda973-4a83-4b1e-8370-4264650a6ecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:16.584336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acda973-4a83-4b1e-8370-4264650a6ecb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:16.597950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 109/215] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:16.597950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3926a-a8d6-44d2-9943-e86f2422150d None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:16.597950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3926a-a8d6-44d2-9943-e86f2422150d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:16.597950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3926a-a8d6-44d2-9943-e86f2422150d None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:16.597950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 110/216] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:16.602106 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97120ae3-bb15-4f20-b983-4490d08b62c8 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:16.602387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97120ae3-bb15-4f20-b983-4490d08b62c8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:16.602652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97120ae3-bb15-4f20-b983-4490d08b62c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:16.622499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-97120ae3-bb15-4f20-b983-4490d08b62c8 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:16.623925 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-97120ae3-bb15-4f20-b983-4490d08b62c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:16.666931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65941e68-f494-469c-b944-4a6e7f425850 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:16.703046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 111/217] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:16.704430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97120ae3-bb15-4f20-b983-4490d08b62c8 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:16.716511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:16.725680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0921d6-c808-43a8-9072-ea84972864c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:16.726665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0921d6-c808-43a8-9072-ea84972864c6 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:16.727108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0921d6-c808-43a8-9072-ea84972864c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:16.727559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0921d6-c808-43a8-9072-ea84972864c6 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:16.728808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0921d6-c808-43a8-9072-ea84972864c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:16.731445 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0921d6-c808-43a8-9072-ea84972864c6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:16.776703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 107/218] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:04:16 2022] POST /identity/v3/users => generated 297 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:04:16.808372 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:16.855197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97120ae3-bb15-4f20-b983-4490d08b62c8 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:16.898337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae340cad-1c5a-4f84-ba9a-048b0121ac1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:16.899018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae340cad-1c5a-4f84-ba9a-048b0121ac1a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:16.899293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae340cad-1c5a-4f84-ba9a-048b0121ac1a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:16.899559 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae340cad-1c5a-4f84-ba9a-048b0121ac1a None admin] PATH_INFO: `/v3/users/ca0ac49016c2446fa93a38c25a584d64` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:16.917257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae340cad-1c5a-4f84-ba9a-048b0121ac1a None admin] RBAC: Authorizing `identity:get_user(user_id=ca0ac49016c2446fa93a38c25a584d64)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:16.918876 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae340cad-1c5a-4f84-ba9a-048b0121ac1a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:16.920767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 112/219] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:16 2022] GET /identity/v3/users/ca0ac49016c2446fa93a38c25a584d64 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:16.926947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0921d6-c808-43a8-9072-ea84972864c6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:17.000669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae340cad-1c5a-4f84-ba9a-048b0121ac1a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:17.046712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0921d6-c808-43a8-9072-ea84972864c6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:17.091299 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef0500-cb6a-49f9-9d37-ce902a005bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:17.092469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef0500-cb6a-49f9-9d37-ce902a005bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:17.092469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef0500-cb6a-49f9-9d37-ce902a005bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:17.092469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef0500-cb6a-49f9-9d37-ce902a005bf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:17.125761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae340cad-1c5a-4f84-ba9a-048b0121ac1a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:17.163904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:17.166005 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:17.172348 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c4d34-15ac-4b8a-8ade-7829c246342a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:17.173859 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c4d34-15ac-4b8a-8ade-7829c246342a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:17.174137 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c4d34-15ac-4b8a-8ade-7829c246342a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:17.174403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c4d34-15ac-4b8a-8ade-7829c246342a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:17.183725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef0500-cb6a-49f9-9d37-ce902a005bf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:17.185278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef0500-cb6a-49f9-9d37-ce902a005bf6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:17.192643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c4d34-15ac-4b8a-8ade-7829c246342a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-3d7c4d34-15ac-4b8a-8ade-7829c246342a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:17.195324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:17.196221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:17.197148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:17.197148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:17.197148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:17.197148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:17.197148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:17.197148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:17.197148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:17.197148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:17.197148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:17.197148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:17.197148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:17.197148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:17.197148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 113/220] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:04:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:04:17.198635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef388ee5-c873-4db9-8cfb-545c22184781 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:17.198799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef388ee5-c873-4db9-8cfb-545c22184781 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:17.200623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef388ee5-c873-4db9-8cfb-545c22184781 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:17.201067 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 114/221] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:17.206225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b88337-1d87-4923-a2a8-365ec932da97 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:17.206501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b88337-1d87-4923-a2a8-365ec932da97 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:17.206766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b88337-1d87-4923-a2a8-365ec932da97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:17.226065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b2b88337-1d87-4923-a2a8-365ec932da97 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:17.227223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b2b88337-1d87-4923-a2a8-365ec932da97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:17.300412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 115/222] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:17.301134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b88337-1d87-4923-a2a8-365ec932da97 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:17.325355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 108/223] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:17.331064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef0500-cb6a-49f9-9d37-ce902a005bf6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:17.491206 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:17.499500 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b88337-1d87-4923-a2a8-365ec932da97 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:17.521539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef0500-cb6a-49f9-9d37-ce902a005bf6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:17.561003 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:17.572198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba9b71-9b9d-4d2b-8d52-7648e65297d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:17.573141 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba9b71-9b9d-4d2b-8d52-7648e65297d0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:17.573440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba9b71-9b9d-4d2b-8d52-7648e65297d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:17.573711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba9b71-9b9d-4d2b-8d52-7648e65297d0 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:17.576645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba9b71-9b9d-4d2b-8d52-7648e65297d0 None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:17.578404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba9b71-9b9d-4d2b-8d52-7648e65297d0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:17.585974 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 109/224] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:17 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:17.586932 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba9b71-9b9d-4d2b-8d52-7648e65297d0 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:17.593987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aca8c2-11a2-4819-a0eb-2713090241d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:17.594659 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aca8c2-11a2-4819-a0eb-2713090241d6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:17.594926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aca8c2-11a2-4819-a0eb-2713090241d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:17.595190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aca8c2-11a2-4819-a0eb-2713090241d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:17.595871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aca8c2-11a2-4819-a0eb-2713090241d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:17.596166 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aca8c2-11a2-4819-a0eb-2713090241d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:17.597594 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aca8c2-11a2-4819-a0eb-2713090241d6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:17.619152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 116/225] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:17.625860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aca8c2-11a2-4819-a0eb-2713090241d6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:17.731404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba9b71-9b9d-4d2b-8d52-7648e65297d0 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:17.764368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aca8c2-11a2-4819-a0eb-2713090241d6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:17.770145 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e99b5-b3a8-4811-bb69-06df4dd21a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:17.771055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e99b5-b3a8-4811-bb69-06df4dd21a23 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:17.771587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e99b5-b3a8-4811-bb69-06df4dd21a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:17.772056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e99b5-b3a8-4811-bb69-06df4dd21a23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:17.773174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e99b5-b3a8-4811-bb69-06df4dd21a23 None admin] RBAC: Adding query filter params (role.id=2921a7146fd9408396ca98a4be4ecd3f, scope.project.id=117d6fb504534ef2bd643528f241dffa, user.id=ca0ac49016c2446fa93a38c25a584d64) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:17.773679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e99b5-b3a8-4811-bb69-06df4dd21a23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:17.775429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e99b5-b3a8-4811-bb69-06df4dd21a23 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:17.782856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 110/226] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:17 2022] GET /identity/v3/role_assignments?user.id=ca0ac49016c2446fa93a38c25a584d64&scope.project.id=117d6fb504534ef2bd643528f241dffa&role.id=2921a7146fd9408396ca98a4be4ecd3f => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:17.784152 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e99b5-b3a8-4811-bb69-06df4dd21a23 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:17.812102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dff2bf-2ea2-4787-affa-a1c768fcf8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:17.814269 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dff2bf-2ea2-4787-affa-a1c768fcf8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:17.818021 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dff2bf-2ea2-4787-affa-a1c768fcf8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:17.818496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:17.819775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dff2bf-2ea2-4787-affa-a1c768fcf8d0 None admin] PATH_INFO: `/v3/users/20a038652c604d3db04c261857dd4e93` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:17.822640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dff2bf-2ea2-4787-affa-a1c768fcf8d0 None admin] RBAC: Authorizing `identity:get_user(user_id=20a038652c604d3db04c261857dd4e93)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:17.825968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dff2bf-2ea2-4787-affa-a1c768fcf8d0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:17.830247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 117/227] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:17 2022] GET /identity/v3/users/20a038652c604d3db04c261857dd4e93 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:17.834908 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dff2bf-2ea2-4787-affa-a1c768fcf8d0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:17.935307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:18.004046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e99b5-b3a8-4811-bb69-06df4dd21a23 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:18.031061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dff2bf-2ea2-4787-affa-a1c768fcf8d0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:18.060267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b1564-7ef8-4257-b307-a052315a4631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:18.061333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b1564-7ef8-4257-b307-a052315a4631 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:18.061971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b1564-7ef8-4257-b307-a052315a4631 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:18.062805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b1564-7ef8-4257-b307-a052315a4631 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:18.077027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b1564-7ef8-4257-b307-a052315a4631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:18.088642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745dd4a-85b7-4d20-a5ef-3206e11e43d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:18.088642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745dd4a-85b7-4d20-a5ef-3206e11e43d7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:18.088642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745dd4a-85b7-4d20-a5ef-3206e11e43d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:18.088642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745dd4a-85b7-4d20-a5ef-3206e11e43d7 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:18.089034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745dd4a-85b7-4d20-a5ef-3206e11e43d7 None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:18.089034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745dd4a-85b7-4d20-a5ef-3206e11e43d7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:18.089667 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 118/228] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:17 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-5d6b1564-7ef8-4257-b307-a052315a4631 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:18.092947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:18.093797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:18.094670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:18.094670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:18.094670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:18.094670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:18.094670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:18.094670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:18.094670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:18.094670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:18.094670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:18.094670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:18.094670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:18.094670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:18.095710 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745dd4a-85b7-4d20-a5ef-3206e11e43d7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:18.097180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 111/229] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:04:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:18.102909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b1564-7ef8-4257-b307-a052315a4631 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:18.251421 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:18.252890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745dd4a-85b7-4d20-a5ef-3206e11e43d7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:18.290874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:18.296398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b1564-7ef8-4257-b307-a052315a4631 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:18.334865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24a27e-d673-4483-8348-45c372ac9057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:18.335461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24a27e-d673-4483-8348-45c372ac9057 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:18.335738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24a27e-d673-4483-8348-45c372ac9057 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:18.336007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24a27e-d673-4483-8348-45c372ac9057 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa/users/20a038652c604d3db04c261857dd4e93/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:18.338763 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24a27e-d673-4483-8348-45c372ac9057 None admin] RBAC: Authorizing `identity:create_grant(project_id=117d6fb504534ef2bd643528f241dffa, user_id=20a038652c604d3db04c261857dd4e93, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:18.340486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa5847-88ba-4b7e-8292-a85d0e290d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:18.341224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa5847-88ba-4b7e-8292-a85d0e290d01 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:18.341497 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa5847-88ba-4b7e-8292-a85d0e290d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:18.341762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa5847-88ba-4b7e-8292-a85d0e290d01 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:18.342443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa5847-88ba-4b7e-8292-a85d0e290d01 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:18.342739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa5847-88ba-4b7e-8292-a85d0e290d01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:18.344113 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa5847-88ba-4b7e-8292-a85d0e290d01 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:18.346348 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24a27e-d673-4483-8348-45c372ac9057 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:18.365337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 112/230] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:04:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:18.366390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 119/231] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:04:18 2022] PUT /identity/v3/projects/117d6fb504534ef2bd643528f241dffa/users/20a038652c604d3db04c261857dd4e93/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:18.371288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa5847-88ba-4b7e-8292-a85d0e290d01 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:18.512939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa5847-88ba-4b7e-8292-a85d0e290d01 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:18.563098 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784a88c-55cc-47ae-aaeb-40e64b42a858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:18.563765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784a88c-55cc-47ae-aaeb-40e64b42a858 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:18.564038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784a88c-55cc-47ae-aaeb-40e64b42a858 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:18.564305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784a88c-55cc-47ae-aaeb-40e64b42a858 None admin] PATH_INFO: `/v3/users/20a038652c604d3db04c261857dd4e93` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:18.565485 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784a88c-55cc-47ae-aaeb-40e64b42a858 None admin] RBAC: Authorizing `identity:get_user(user_id=20a038652c604d3db04c261857dd4e93)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:18.566911 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a784a88c-55cc-47ae-aaeb-40e64b42a858 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:18.568781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 113/232] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:18 2022] GET /identity/v3/users/20a038652c604d3db04c261857dd4e93 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:18.574961 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24a27e-d673-4483-8348-45c372ac9057 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:18.703691 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24a27e-d673-4483-8348-45c372ac9057 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:18.748649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7b01c-d6dd-4ab2-ae23-f4eb38ce1980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:18.749651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7b01c-d6dd-4ab2-ae23-f4eb38ce1980 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:18.750124 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7b01c-d6dd-4ab2-ae23-f4eb38ce1980 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:18.750664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7b01c-d6dd-4ab2-ae23-f4eb38ce1980 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:18.752077 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7b01c-d6dd-4ab2-ae23-f4eb38ce1980 None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:18.753888 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7b01c-d6dd-4ab2-ae23-f4eb38ce1980 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:18.755928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 120/233] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:18 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:18.763567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784a88c-55cc-47ae-aaeb-40e64b42a858 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:18.788986 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:18.917537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784a88c-55cc-47ae-aaeb-40e64b42a858 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:18.958727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d27c6-d4f9-485b-a6f4-ac24b2f3eba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:18.959319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d27c6-d4f9-485b-a6f4-ac24b2f3eba6 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:18.959587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d27c6-d4f9-485b-a6f4-ac24b2f3eba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:18.959852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d27c6-d4f9-485b-a6f4-ac24b2f3eba6 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa/users/20a038652c604d3db04c261857dd4e93/roles/704c67b455b54fb48089faef4aba8ccd` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:18.962128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d27c6-d4f9-485b-a6f4-ac24b2f3eba6 None admin] RBAC: Authorizing `identity:create_grant(project_id=117d6fb504534ef2bd643528f241dffa, user_id=20a038652c604d3db04c261857dd4e93, role_id=704c67b455b54fb48089faef4aba8ccd)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:18.963639 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d27c6-d4f9-485b-a6f4-ac24b2f3eba6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:18.987668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 114/234] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:04:18 2022] PUT /identity/v3/projects/117d6fb504534ef2bd643528f241dffa/users/20a038652c604d3db04c261857dd4e93/roles/704c67b455b54fb48089faef4aba8ccd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:19.541207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5b8e-0ed2-44de-a395-2ec78d21d557 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:19.541741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5b8e-0ed2-44de-a395-2ec78d21d557 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:19.541741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5b8e-0ed2-44de-a395-2ec78d21d557 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:19.543157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 121/235] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:19.549863 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf535b9-1eb3-46cc-a7ed-1636771bae14 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:19.550290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf535b9-1eb3-46cc-a7ed-1636771bae14 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:19.550598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf535b9-1eb3-46cc-a7ed-1636771bae14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:19.569693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8cf535b9-1eb3-46cc-a7ed-1636771bae14 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:19.570902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8cf535b9-1eb3-46cc-a7ed-1636771bae14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:19.640095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:19.645418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 115/236] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:19.804570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c6c57-b0fa-499d-b2c0-010539e275bb None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:19.804809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c6c57-b0fa-499d-b2c0-010539e275bb None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:19.805089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c6c57-b0fa-499d-b2c0-010539e275bb None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:19.806681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 122/237] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:19.812558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71feed-4ee6-4560-8ac8-8ff28cc799fe None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:19.813180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71feed-4ee6-4560-8ac8-8ff28cc799fe None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:19.813669 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71feed-4ee6-4560-8ac8-8ff28cc799fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:19.841092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7d71feed-4ee6-4560-8ac8-8ff28cc799fe None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:19.842811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7d71feed-4ee6-4560-8ac8-8ff28cc799fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:19.931240 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 116/238] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:19.967446 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c6c57-b0fa-499d-b2c0-010539e275bb None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:20.019148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:20.078974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c6c57-b0fa-499d-b2c0-010539e275bb None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:20.122275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab6a6a-228d-44b8-bc85-c4fb0c1f43c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:20.122972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab6a6a-228d-44b8-bc85-c4fb0c1f43c3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:20.123250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab6a6a-228d-44b8-bc85-c4fb0c1f43c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:20.123515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab6a6a-228d-44b8-bc85-c4fb0c1f43c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:20.138976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab6a6a-228d-44b8-bc85-c4fb0c1f43c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-ccab6a6a-228d-44b8-bc85-c4fb0c1f43c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:20.146258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:20.147143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:20.147923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:20.147923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:20.147923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:20.147923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:20.147923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:20.147923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:20.147923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:20.147923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:20.147923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:20.147923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:20.147923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:20.147923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:20.147923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 123/239] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:20.156040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71feed-4ee6-4560-8ac8-8ff28cc799fe None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:20.294766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71feed-4ee6-4560-8ac8-8ff28cc799fe None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:20.316513 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab6a6a-228d-44b8-bc85-c4fb0c1f43c3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:20.337882 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:20.347454 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce46fd7-5791-4cd6-9b58-8ec0eb17995b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:20.348404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce46fd7-5791-4cd6-9b58-8ec0eb17995b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:20.348707 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce46fd7-5791-4cd6-9b58-8ec0eb17995b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:20.349115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce46fd7-5791-4cd6-9b58-8ec0eb17995b None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:20.349992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce46fd7-5791-4cd6-9b58-8ec0eb17995b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:20.350294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce46fd7-5791-4cd6-9b58-8ec0eb17995b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:20.351760 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce46fd7-5791-4cd6-9b58-8ec0eb17995b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:20.371155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 117/240] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:20.376682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce46fd7-5791-4cd6-9b58-8ec0eb17995b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:20.462757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab6a6a-228d-44b8-bc85-c4fb0c1f43c3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:20.470148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:20.509939 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534469f-a9d3-4822-aa18-60635108e7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:20.510906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534469f-a9d3-4822-aa18-60635108e7be None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:20.511324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534469f-a9d3-4822-aa18-60635108e7be None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:20.511737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534469f-a9d3-4822-aa18-60635108e7be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:20.513123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534469f-a9d3-4822-aa18-60635108e7be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:20.514755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534469f-a9d3-4822-aa18-60635108e7be None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:20.515877 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce46fd7-5791-4cd6-9b58-8ec0eb17995b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:20.516693 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 124/241] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:20.527188 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534469f-a9d3-4822-aa18-60635108e7be None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:20.574689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ad8d6-45e6-4215-a5b6-d82564ec1cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:20.575961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ad8d6-45e6-4215-a5b6-d82564ec1cdc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:20.576377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ad8d6-45e6-4215-a5b6-d82564ec1cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:20.576884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ad8d6-45e6-4215-a5b6-d82564ec1cdc None admin] PATH_INFO: `/v3/users/8f7ad21310fa43fbbb36e1ca299cf471` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:20.610009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ad8d6-45e6-4215-a5b6-d82564ec1cdc None admin] RBAC: Authorizing `identity:get_user(user_id=8f7ad21310fa43fbbb36e1ca299cf471)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:20.611866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ad8d6-45e6-4215-a5b6-d82564ec1cdc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:20.617221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 118/242] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:20 2022] GET /identity/v3/users/8f7ad21310fa43fbbb36e1ca299cf471 => generated 297 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:20.621259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ad8d6-45e6-4215-a5b6-d82564ec1cdc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:20.729943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534469f-a9d3-4822-aa18-60635108e7be None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:20.770943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246787e-f517-4b57-a256-1a95d65f3713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:20.771809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246787e-f517-4b57-a256-1a95d65f3713 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:20.772177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246787e-f517-4b57-a256-1a95d65f3713 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:20.772553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246787e-f517-4b57-a256-1a95d65f3713 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:20.773312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:20.773481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246787e-f517-4b57-a256-1a95d65f3713 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:20.775477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6246787e-f517-4b57-a256-1a95d65f3713 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:20.804314 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 125/243] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:04:20 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:04:20.812171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ce110-aff2-4655-ab19-b72cdd85a209 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:20.812724 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ce110-aff2-4655-ab19-b72cdd85a209 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:20.813133 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ce110-aff2-4655-ab19-b72cdd85a209 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:20.817461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 126/244] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:20 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:20.822523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585a925-0404-4b7e-b4c6-f572399e3258 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:20.823138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585a925-0404-4b7e-b4c6-f572399e3258 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:20.823550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585a925-0404-4b7e-b4c6-f572399e3258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:20.860865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e585a925-0404-4b7e-b4c6-f572399e3258 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:20.863081 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e585a925-0404-4b7e-b4c6-f572399e3258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:20.868107 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ad8d6-45e6-4215-a5b6-d82564ec1cdc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:20.920979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:20.937328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e0d22f-3aa3-4526-805a-1b086442f3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:20.940504 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0d22f-3aa3-4526-805a-1b086442f3a0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:20.941035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0d22f-3aa3-4526-805a-1b086442f3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:20.941444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0d22f-3aa3-4526-805a-1b086442f3a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:21.010503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 127/245] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:21.056190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0d22f-3aa3-4526-805a-1b086442f3a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:21.058046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e0d22f-3aa3-4526-805a-1b086442f3a0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:21.122537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:21.170780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61187a7-93e8-459d-8fce-608a02653ec9 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:21.171127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61187a7-93e8-459d-8fce-608a02653ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:21.171316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61187a7-93e8-459d-8fce-608a02653ec9 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:21.174588 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 128/246] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:21.176016 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 119/247] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:21.179140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18e216-4920-4a7a-a4fa-33a4257f6e0b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:21.179647 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18e216-4920-4a7a-a4fa-33a4257f6e0b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:21.180131 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18e216-4920-4a7a-a4fa-33a4257f6e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:21.181573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e0d22f-3aa3-4526-805a-1b086442f3a0 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:21.209926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5b18e216-4920-4a7a-a4fa-33a4257f6e0b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:21.211583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5b18e216-4920-4a7a-a4fa-33a4257f6e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:21.304663 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 129/248] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:21.316536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e0d22f-3aa3-4526-805a-1b086442f3a0 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:21.356013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043c931-942d-45f1-857b-10d63d591744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:21.357090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043c931-942d-45f1-857b-10d63d591744 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:21.357562 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043c931-942d-45f1-857b-10d63d591744 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:21.358376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043c931-942d-45f1-857b-10d63d591744 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:21.359783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043c931-942d-45f1-857b-10d63d591744 None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:21.361489 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043c931-942d-45f1-857b-10d63d591744 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:21.363519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 120/249] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:21 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:21.364056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18e216-4920-4a7a-a4fa-33a4257f6e0b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:21.375447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043c931-942d-45f1-857b-10d63d591744 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:21.406501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:21.528424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:21.552616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18e216-4920-4a7a-a4fa-33a4257f6e0b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:21.572438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043c931-942d-45f1-857b-10d63d591744 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:21.617350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e8110-1a1a-4d7f-8c8e-5454cd5f95d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:21.618051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e8110-1a1a-4d7f-8c8e-5454cd5f95d8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:21.618334 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e8110-1a1a-4d7f-8c8e-5454cd5f95d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:21.618449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180851e6-9e71-49b9-abb2-5e45a57a8118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:21.618875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e8110-1a1a-4d7f-8c8e-5454cd5f95d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:21.619059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180851e6-9e71-49b9-abb2-5e45a57a8118 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:21.619405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180851e6-9e71-49b9-abb2-5e45a57a8118 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:21.619704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180851e6-9e71-49b9-abb2-5e45a57a8118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:21.620640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180851e6-9e71-49b9-abb2-5e45a57a8118 None admin] RBAC: Adding query filter params (role.id=712114868a39430b80ec9c19cb91c947, scope.project.id=117d6fb504534ef2bd643528f241dffa, user.id=8f7ad21310fa43fbbb36e1ca299cf471) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:21.620953 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180851e6-9e71-49b9-abb2-5e45a57a8118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:21.622572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180851e6-9e71-49b9-abb2-5e45a57a8118 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:21.630604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 121/250] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:21 2022] GET /identity/v3/role_assignments?user.id=8f7ad21310fa43fbbb36e1ca299cf471&scope.project.id=117d6fb504534ef2bd643528f241dffa&role.id=712114868a39430b80ec9c19cb91c947 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:21.631397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180851e6-9e71-49b9-abb2-5e45a57a8118 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:21.639876 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e8110-1a1a-4d7f-8c8e-5454cd5f95d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-553e8110-1a1a-4d7f-8c8e-5454cd5f95d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:21.646160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:21.647267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:21.648230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:21.648230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:21.648230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:21.648230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:21.648230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:21.648230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:21.648230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:21.648230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:21.648230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:21.648230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:21.648230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:21.648230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:21.650239 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 130/251] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:04:21.657115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e8110-1a1a-4d7f-8c8e-5454cd5f95d8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:21.792194 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:21.848287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180851e6-9e71-49b9-abb2-5e45a57a8118 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:21.891792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e8110-1a1a-4d7f-8c8e-5454cd5f95d8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:21.925105 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8b987-7a51-4ce5-89ec-a6c3a59e76bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:21.925749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8b987-7a51-4ce5-89ec-a6c3a59e76bc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:21.926856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8b987-7a51-4ce5-89ec-a6c3a59e76bc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:21.927126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8b987-7a51-4ce5-89ec-a6c3a59e76bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:21.953631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8b987-7a51-4ce5-89ec-a6c3a59e76bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-ddf8b987-7a51-4ce5-89ec-a6c3a59e76bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:21.959465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:21.961224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 122/252] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:04:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:21.962749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bb5fc-d271-45bf-93d0-05f04ef8ba61 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:21.966512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bb5fc-d271-45bf-93d0-05f04ef8ba61 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:21.966512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bb5fc-d271-45bf-93d0-05f04ef8ba61 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:21.966512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 123/253] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:21.971647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bb5fc-d271-45bf-93d0-05f04ef8ba61 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:21.994283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ea201-f54c-47a7-a1d2-3e3cc6c858d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:21.995598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ea201-f54c-47a7-a1d2-3e3cc6c858d4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:21.995598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ea201-f54c-47a7-a1d2-3e3cc6c858d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:21.995598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ea201-f54c-47a7-a1d2-3e3cc6c858d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:21.996313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ea201-f54c-47a7-a1d2-3e3cc6c858d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:21.999842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ea201-f54c-47a7-a1d2-3e3cc6c858d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:21.999842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ea201-f54c-47a7-a1d2-3e3cc6c858d4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:22.024468 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 131/254] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:22.026014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929edb0-0ae2-4cd7-b9e0-31307313dea9 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:22.026750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929edb0-0ae2-4cd7-b9e0-31307313dea9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:22.026750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b929edb0-0ae2-4cd7-b9e0-31307313dea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:22.070436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b929edb0-0ae2-4cd7-b9e0-31307313dea9 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:22.071657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b929edb0-0ae2-4cd7-b9e0-31307313dea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:22.088275 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bb5fc-d271-45bf-93d0-05f04ef8ba61 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:22.091442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:22.126976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256259f-fc8a-49b6-a952-36e2d3f28b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:22.127576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256259f-fc8a-49b6-a952-36e2d3f28b56 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:22.127885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256259f-fc8a-49b6-a952-36e2d3f28b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:22.136732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256259f-fc8a-49b6-a952-36e2d3f28b56 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:22.136732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256259f-fc8a-49b6-a952-36e2d3f28b56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:22.136732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256259f-fc8a-49b6-a952-36e2d3f28b56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:22.136732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256259f-fc8a-49b6-a952-36e2d3f28b56 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:22.140702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 124/255] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:22.141362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256259f-fc8a-49b6-a952-36e2d3f28b56 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:22.151519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:22.238959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 132/256] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:22.239838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929edb0-0ae2-4cd7-b9e0-31307313dea9 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:22.263045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256259f-fc8a-49b6-a952-36e2d3f28b56 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:22.299849 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7ea6a-f01f-41fd-995a-f26ea450cd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:22.300481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7ea6a-f01f-41fd-995a-f26ea450cd92 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:22.300909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7ea6a-f01f-41fd-995a-f26ea450cd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:22.301181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7ea6a-f01f-41fd-995a-f26ea450cd92 None admin] PATH_INFO: `/v3/users/ca0ac49016c2446fa93a38c25a584d64` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:22.302352 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7ea6a-f01f-41fd-995a-f26ea450cd92 None admin] RBAC: Authorizing `identity:get_user(user_id=ca0ac49016c2446fa93a38c25a584d64)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:22.303787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7ea6a-f01f-41fd-995a-f26ea450cd92 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:22.305518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 125/257] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:22 2022] GET /identity/v3/users/ca0ac49016c2446fa93a38c25a584d64 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:22.312738 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7ea6a-f01f-41fd-995a-f26ea450cd92 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:22.392982 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:22.427947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7ea6a-f01f-41fd-995a-f26ea450cd92 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:22.467116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376ce42-a5bb-47bd-981e-73e937fa3763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:22.467758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376ce42-a5bb-47bd-981e-73e937fa3763 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:22.468025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376ce42-a5bb-47bd-981e-73e937fa3763 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:22.468289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376ce42-a5bb-47bd-981e-73e937fa3763 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:22.469963 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376ce42-a5bb-47bd-981e-73e937fa3763 None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:22.471641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376ce42-a5bb-47bd-981e-73e937fa3763 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:22.473664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 126/258] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:22 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:22.478867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b929edb0-0ae2-4cd7-b9e0-31307313dea9 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:22.479523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376ce42-a5bb-47bd-981e-73e937fa3763 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:22.573957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca3fd2-44f0-43f5-8be8-055c14cc476b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:22.574616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca3fd2-44f0-43f5-8be8-055c14cc476b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:22.574886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca3fd2-44f0-43f5-8be8-055c14cc476b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:22.575153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbca3fd2-44f0-43f5-8be8-055c14cc476b None admin] PATH_INFO: `/v3/users/20a038652c604d3db04c261857dd4e93` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:22.577305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca3fd2-44f0-43f5-8be8-055c14cc476b None admin] RBAC: Authorizing `identity:get_user(user_id=20a038652c604d3db04c261857dd4e93)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:22.578762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbca3fd2-44f0-43f5-8be8-055c14cc476b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:22.580397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 133/259] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:22 2022] GET /identity/v3/users/20a038652c604d3db04c261857dd4e93 => generated 283 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:22.593403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376ce42-a5bb-47bd-981e-73e937fa3763 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:22.601637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca3fd2-44f0-43f5-8be8-055c14cc476b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:22.631516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c59b8-8abf-4f63-b7a0-e9ff622eb449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:22.632933 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:22.633200 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c59b8-8abf-4f63-b7a0-e9ff622eb449 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:22.633475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c59b8-8abf-4f63-b7a0-e9ff622eb449 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:22.633736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c59b8-8abf-4f63-b7a0-e9ff622eb449 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa/users/ca0ac49016c2446fa93a38c25a584d64/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:22.638632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c59b8-8abf-4f63-b7a0-e9ff622eb449 None admin] RBAC: Authorizing `identity:create_grant(project_id=117d6fb504534ef2bd643528f241dffa, user_id=ca0ac49016c2446fa93a38c25a584d64, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:22.642539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c59b8-8abf-4f63-b7a0-e9ff622eb449 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:22.659211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:22.669086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 127/260] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:04:22 2022] PUT /identity/v3/projects/117d6fb504534ef2bd643528f241dffa/users/ca0ac49016c2446fa93a38c25a584d64/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:22.772905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbca3fd2-44f0-43f5-8be8-055c14cc476b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:22.793910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c59b8-8abf-4f63-b7a0-e9ff622eb449 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:22.824407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0244087-a102-4c92-a27d-3eb96f2725f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:22.825425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0244087-a102-4c92-a27d-3eb96f2725f6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:22.825713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0244087-a102-4c92-a27d-3eb96f2725f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:22.825976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0244087-a102-4c92-a27d-3eb96f2725f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:22.989545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0244087-a102-4c92-a27d-3eb96f2725f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:22.991872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0244087-a102-4c92-a27d-3eb96f2725f6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:23.017035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:23.066316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c59b8-8abf-4f63-b7a0-e9ff622eb449 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:23.144054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ed067-f72b-44b1-9ad4-d31e9c671270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:23.144721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ed067-f72b-44b1-9ad4-d31e9c671270 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:23.144997 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ed067-f72b-44b1-9ad4-d31e9c671270 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:23.145263 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ed067-f72b-44b1-9ad4-d31e9c671270 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:23.171438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ed067-f72b-44b1-9ad4-d31e9c671270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-438ed067-f72b-44b1-9ad4-d31e9c671270 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:23.183309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:23.184248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:23.185049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:23.185049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:23.185049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:23.185049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:23.185049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:23.185049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:23.185049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:23.185049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:23.185049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:23.185049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:23.185049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:23.185049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:23.185642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 128/261] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:04:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:04:23.191495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ed067-f72b-44b1-9ad4-d31e9c671270 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:23.234516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 134/262] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:23.241243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0244087-a102-4c92-a27d-3eb96f2725f6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:23.306157 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:23.372834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ed067-f72b-44b1-9ad4-d31e9c671270 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:23.430764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b4655-590c-442b-8725-fa584ae97a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:23.431386 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b4655-590c-442b-8725-fa584ae97a11 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:23.431729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b4655-590c-442b-8725-fa584ae97a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:23.432001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b4655-590c-442b-8725-fa584ae97a11 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:23.432705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b4655-590c-442b-8725-fa584ae97a11 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:23.433010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b4655-590c-442b-8725-fa584ae97a11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:23.434509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b4655-590c-442b-8725-fa584ae97a11 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:23.451781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 129/263] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:04:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:23.457926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b4655-590c-442b-8725-fa584ae97a11 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:23.507286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0244087-a102-4c92-a27d-3eb96f2725f6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:23.524721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:23.601312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b1ef5-784e-4ea1-986e-e257da2c0a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:23.601965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b1ef5-784e-4ea1-986e-e257da2c0a3d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:23.602239 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b1ef5-784e-4ea1-986e-e257da2c0a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:23.602504 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b1ef5-784e-4ea1-986e-e257da2c0a3d None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:23.603890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b1ef5-784e-4ea1-986e-e257da2c0a3d None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:23.605899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b1ef5-784e-4ea1-986e-e257da2c0a3d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:23.607917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 135/264] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:23 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:23.614662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b1ef5-784e-4ea1-986e-e257da2c0a3d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:23.692353 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b4655-590c-442b-8725-fa584ae97a11 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:23.739742 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:23.751218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfe7f4-8604-4458-9a52-a3b7a60a13d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:23.752223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfe7f4-8604-4458-9a52-a3b7a60a13d9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:23.752750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfe7f4-8604-4458-9a52-a3b7a60a13d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:23.753314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfe7f4-8604-4458-9a52-a3b7a60a13d9 None admin] PATH_INFO: `/v3/users/20a038652c604d3db04c261857dd4e93` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:23.754654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfe7f4-8604-4458-9a52-a3b7a60a13d9 None admin] RBAC: Authorizing `identity:get_user(user_id=20a038652c604d3db04c261857dd4e93)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:23.756608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfe7f4-8604-4458-9a52-a3b7a60a13d9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:23.759252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 130/265] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:23 2022] GET /identity/v3/users/20a038652c604d3db04c261857dd4e93 => generated 283 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:23.765858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfe7f4-8604-4458-9a52-a3b7a60a13d9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:23.914136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b1ef5-784e-4ea1-986e-e257da2c0a3d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:23.930756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfe7f4-8604-4458-9a52-a3b7a60a13d9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:23.968223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5ba3c-48bd-4e8a-b510-60a42d091703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:23.969287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5ba3c-48bd-4e8a-b510-60a42d091703 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:23.969749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5ba3c-48bd-4e8a-b510-60a42d091703 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:23.970203 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5ba3c-48bd-4e8a-b510-60a42d091703 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:23.995737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f630f-81b4-45d1-97ab-34f6d8f92884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:23.996512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f630f-81b4-45d1-97ab-34f6d8f92884 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:23.996610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f630f-81b4-45d1-97ab-34f6d8f92884 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:23.996845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f630f-81b4-45d1-97ab-34f6d8f92884 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:23.997680 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f630f-81b4-45d1-97ab-34f6d8f92884 None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=117d6fb504534ef2bd643528f241dffa, user.id=20a038652c604d3db04c261857dd4e93) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:23.997864 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f630f-81b4-45d1-97ab-34f6d8f92884 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:23.999313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f630f-81b4-45d1-97ab-34f6d8f92884 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:24.012517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 136/266] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:23 2022] GET /identity/v3/role_assignments?user.id=20a038652c604d3db04c261857dd4e93&scope.project.id=117d6fb504534ef2bd643528f241dffa&role.id=ae444a489623434999d758b71ae2dc87 => generated 608 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:24.053302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:24.063594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5ba3c-48bd-4e8a-b510-60a42d091703 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:24.065129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5ba3c-48bd-4e8a-b510-60a42d091703 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:24.176222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 131/267] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 15:04:24.182736 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f630f-81b4-45d1-97ab-34f6d8f92884 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:24.268907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:24.355589 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f630f-81b4-45d1-97ab-34f6d8f92884 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:24.424925 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66e8cf-df4b-439e-9d80-24214cd63aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:24.425591 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66e8cf-df4b-439e-9d80-24214cd63aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:24.425862 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66e8cf-df4b-439e-9d80-24214cd63aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:24.426126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66e8cf-df4b-439e-9d80-24214cd63aa2 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:24.433619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66e8cf-df4b-439e-9d80-24214cd63aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:24.435132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66e8cf-df4b-439e-9d80-24214cd63aa2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:24.437146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 137/268] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:24 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:24.443130 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5ba3c-48bd-4e8a-b510-60a42d091703 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:24.574051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:24.586903 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5ba3c-48bd-4e8a-b510-60a42d091703 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:24.631702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac3882-d830-4a06-a317-adcc8a742b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:24.632312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac3882-d830-4a06-a317-adcc8a742b76 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:24.632582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac3882-d830-4a06-a317-adcc8a742b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:24.632908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac3882-d830-4a06-a317-adcc8a742b76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:24.633640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac3882-d830-4a06-a317-adcc8a742b76 None admin] RBAC: Adding query filter params (role.id=704c67b455b54fb48089faef4aba8ccd, scope.project.id=117d6fb504534ef2bd643528f241dffa, user.id=20a038652c604d3db04c261857dd4e93) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:24.633934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac3882-d830-4a06-a317-adcc8a742b76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:24.635544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac3882-d830-4a06-a317-adcc8a742b76 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:24.642287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 132/269] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:24 2022] GET /identity/v3/role_assignments?user.id=20a038652c604d3db04c261857dd4e93&scope.project.id=117d6fb504534ef2bd643528f241dffa&role.id=704c67b455b54fb48089faef4aba8ccd => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:24.656840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671dbc0-d818-453b-bef2-db1c2fca5629 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:24.657102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671dbc0-d818-453b-bef2-db1c2fca5629 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:24.657442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671dbc0-d818-453b-bef2-db1c2fca5629 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:24.658760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 138/270] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:24.667472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65299f76-1763-42a0-b457-0d1b658e2457 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:24.667859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65299f76-1763-42a0-b457-0d1b658e2457 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:24.667999 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65299f76-1763-42a0-b457-0d1b658e2457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:24.688004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-65299f76-1763-42a0-b457-0d1b658e2457 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:24.689240 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-65299f76-1763-42a0-b457-0d1b658e2457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:24.763294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 133/271] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:24.807143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c1008-ed6c-47e3-bacc-7be46ff3f646 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:24.807143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c1008-ed6c-47e3-bacc-7be46ff3f646 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:24.807418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c1008-ed6c-47e3-bacc-7be46ff3f646 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:24.808651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 139/272] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:24.814796 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a3f42-9e43-4f44-9572-d6d5eff823fa None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:24.815108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a3f42-9e43-4f44-9572-d6d5eff823fa None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:24.815416 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a3f42-9e43-4f44-9572-d6d5eff823fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:24.835717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cc3a3f42-9e43-4f44-9572-d6d5eff823fa None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:24.837012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cc3a3f42-9e43-4f44-9572-d6d5eff823fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:24.921028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 134/273] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:25.023033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f691122-d517-4d08-9f2a-1b2b192d2088 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:25.023536 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f691122-d517-4d08-9f2a-1b2b192d2088 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:25.023536 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f691122-d517-4d08-9f2a-1b2b192d2088 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:25.024926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 140/274] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:25.031680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6270d2-4a00-4c7a-b4bb-3274bfc0f2aa None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:25.031951 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6270d2-4a00-4c7a-b4bb-3274bfc0f2aa None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:25.032221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6270d2-4a00-4c7a-b4bb-3274bfc0f2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:25.053474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5e6270d2-4a00-4c7a-b4bb-3274bfc0f2aa None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:25.054640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5e6270d2-4a00-4c7a-b4bb-3274bfc0f2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:25.064794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:25.080636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f691122-d517-4d08-9f2a-1b2b192d2088 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:25.129227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 135/275] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:25.206901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6270d2-4a00-4c7a-b4bb-3274bfc0f2aa None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:25.215709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f691122-d517-4d08-9f2a-1b2b192d2088 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:25.227012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:25.253651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91557804-3b2c-4e40-a415-dee84e4910d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:25.254281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91557804-3b2c-4e40-a415-dee84e4910d6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:25.254551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91557804-3b2c-4e40-a415-dee84e4910d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:25.254812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91557804-3b2c-4e40-a415-dee84e4910d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:25.270620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91557804-3b2c-4e40-a415-dee84e4910d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-91557804-3b2c-4e40-a415-dee84e4910d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:25.275964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:25.276874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:25.277844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:25.277844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:25.277844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:25.277844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:25.277844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:25.277844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:25.277844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:25.277844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:25.277844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:25.277844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:25.277844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:25.277844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:25.279779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 141/276] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:04:25.285646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91557804-3b2c-4e40-a415-dee84e4910d6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:25.337604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6270d2-4a00-4c7a-b4bb-3274bfc0f2aa None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:25.384478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df434cf3-dca4-4bcb-89bc-f5a202e7f62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:25.386037 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df434cf3-dca4-4bcb-89bc-f5a202e7f62b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:25.386321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df434cf3-dca4-4bcb-89bc-f5a202e7f62b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:25.386590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df434cf3-dca4-4bcb-89bc-f5a202e7f62b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:25.387871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df434cf3-dca4-4bcb-89bc-f5a202e7f62b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:25.389400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df434cf3-dca4-4bcb-89bc-f5a202e7f62b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:25.396099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 136/277] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:25.397174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df434cf3-dca4-4bcb-89bc-f5a202e7f62b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:25.403259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91557804-3b2c-4e40-a415-dee84e4910d6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:25.413519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:25.449387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f807e6d-95eb-42de-81e1-8cecd47f0570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:25.449998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f807e6d-95eb-42de-81e1-8cecd47f0570 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:25.450268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f807e6d-95eb-42de-81e1-8cecd47f0570 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:25.450532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f807e6d-95eb-42de-81e1-8cecd47f0570 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:25.451202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f807e6d-95eb-42de-81e1-8cecd47f0570 None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:25.451500 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f807e6d-95eb-42de-81e1-8cecd47f0570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:25.452917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f807e6d-95eb-42de-81e1-8cecd47f0570 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:25.461467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 142/278] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:25.461467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f807e6d-95eb-42de-81e1-8cecd47f0570 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:25.501426 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:25.521335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df434cf3-dca4-4bcb-89bc-f5a202e7f62b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:25.578542 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdc4f6-d4e8-4509-b6e7-116c273414dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:25.579236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdc4f6-d4e8-4509-b6e7-116c273414dc None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:25.579502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdc4f6-d4e8-4509-b6e7-116c273414dc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:25.579760 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdc4f6-d4e8-4509-b6e7-116c273414dc None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:25.580421 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcdc4f6-d4e8-4509-b6e7-116c273414dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:25.581925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcdc4f6-d4e8-4509-b6e7-116c273414dc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:25.594729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f807e6d-95eb-42de-81e1-8cecd47f0570 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:25.613041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 137/279] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:04:25 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:04:25.614964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdc4f6-d4e8-4509-b6e7-116c273414dc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:25.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fd80d-6cce-4883-866d-e74a92d7489f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:25.647691 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fd80d-6cce-4883-866d-e74a92d7489f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:25.647929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fd80d-6cce-4883-866d-e74a92d7489f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:25.648190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fd80d-6cce-4883-866d-e74a92d7489f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:25.661532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fd80d-6cce-4883-866d-e74a92d7489f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d56fd80d-6cce-4883-866d-e74a92d7489f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:25.668069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:25.669313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:25.670285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:25.670285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:25.670285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:25.670285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:25.670285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:25.670285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:25.670285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:25.670285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:25.670285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:25.670285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:25.670285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:25.670285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:25.670285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 143/280] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:25.678349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fd80d-6cce-4883-866d-e74a92d7489f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:25.722856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:25.777013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdc4f6-d4e8-4509-b6e7-116c273414dc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:25.824508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:25.829729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7daaee-9f61-4630-8850-6e7620e6d22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:25.831111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7daaee-9f61-4630-8850-6e7620e6d22f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:25.831617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7daaee-9f61-4630-8850-6e7620e6d22f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:25.831888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7daaee-9f61-4630-8850-6e7620e6d22f None admin] PATH_INFO: `/v3/users/cd002ecfa3684ab0b1500496760db640` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:25.838409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fd80d-6cce-4883-866d-e74a92d7489f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:25.849229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7daaee-9f61-4630-8850-6e7620e6d22f None admin] RBAC: Authorizing `identity:get_user(user_id=cd002ecfa3684ab0b1500496760db640)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:25.850752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7daaee-9f61-4630-8850-6e7620e6d22f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:25.853600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 138/281] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:25 2022] GET /identity/v3/users/cd002ecfa3684ab0b1500496760db640 => generated 293 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:25.859591 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7daaee-9f61-4630-8850-6e7620e6d22f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:25.879125 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944552ed-093a-4d95-951c-e1e5a9314f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:25.879862 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944552ed-093a-4d95-951c-e1e5a9314f61 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:25.880135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944552ed-093a-4d95-951c-e1e5a9314f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:25.880393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944552ed-093a-4d95-951c-e1e5a9314f61 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:25.881099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944552ed-093a-4d95-951c-e1e5a9314f61 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:25.881392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944552ed-093a-4d95-951c-e1e5a9314f61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:25.882787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944552ed-093a-4d95-951c-e1e5a9314f61 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:25.892776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 144/282] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:25.898340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944552ed-093a-4d95-951c-e1e5a9314f61 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:25.990172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7daaee-9f61-4630-8850-6e7620e6d22f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:26.022690 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944552ed-093a-4d95-951c-e1e5a9314f61 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:26.024271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:26.034428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80b60e-428f-43f3-b1fc-5688570f88b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:26.035039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80b60e-428f-43f3-b1fc-5688570f88b8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:26.035312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80b60e-428f-43f3-b1fc-5688570f88b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:26.035645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80b60e-428f-43f3-b1fc-5688570f88b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:26.036390 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80b60e-428f-43f3-b1fc-5688570f88b8 None admin] RBAC: Adding query filter params (role.id=2921a7146fd9408396ca98a4be4ecd3f, user.id=cd002ecfa3684ab0b1500496760db640, scope.system=all) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:26.036734 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80b60e-428f-43f3-b1fc-5688570f88b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:26.038219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80b60e-428f-43f3-b1fc-5688570f88b8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:26.045498 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 139/283] 10.222.0.75 () {56 vars in 1308 bytes} [Fri Sep 23 15:04:25 2022] GET /identity/v3/role_assignments?user.id=cd002ecfa3684ab0b1500496760db640&scope.system=all&role.id=2921a7146fd9408396ca98a4be4ecd3f => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:26.047287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fc7cf-a79a-40cb-aecf-06356f816335 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:26.047788 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fc7cf-a79a-40cb-aecf-06356f816335 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:26.048331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fc7cf-a79a-40cb-aecf-06356f816335 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:26.049960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 140/284] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:26.056047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f94f28-6790-4c02-8541-605062853a2f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:26.056390 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f94f28-6790-4c02-8541-605062853a2f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:26.056622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f94f28-6790-4c02-8541-605062853a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:26.068922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10adb1-37a2-4f26-814c-4289787c225a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:26.069720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10adb1-37a2-4f26-814c-4289787c225a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:26.070038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10adb1-37a2-4f26-814c-4289787c225a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:26.070305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10adb1-37a2-4f26-814c-4289787c225a None admin] PATH_INFO: `/v3/users/8f7ad21310fa43fbbb36e1ca299cf471` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:26.072430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10adb1-37a2-4f26-814c-4289787c225a None admin] RBAC: Authorizing `identity:get_user(user_id=8f7ad21310fa43fbbb36e1ca299cf471)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:26.074073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10adb1-37a2-4f26-814c-4289787c225a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:26.076531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 145/285] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:25 2022] GET /identity/v3/users/8f7ad21310fa43fbbb36e1ca299cf471 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:26.079010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-74f94f28-6790-4c02-8541-605062853a2f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:26.080253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-74f94f28-6790-4c02-8541-605062853a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:26.082342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10adb1-37a2-4f26-814c-4289787c225a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:26.123587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:26.191062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 141/286] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:26.271710 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10adb1-37a2-4f26-814c-4289787c225a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:26.309841 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f423e6e-a8af-4ca1-bbf5-34f83d452146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:26.310541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f423e6e-a8af-4ca1-bbf5-34f83d452146 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:26.310811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f423e6e-a8af-4ca1-bbf5-34f83d452146 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:26.311075 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f423e6e-a8af-4ca1-bbf5-34f83d452146 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:26.312311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f423e6e-a8af-4ca1-bbf5-34f83d452146 None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:26.313859 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f423e6e-a8af-4ca1-bbf5-34f83d452146 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:26.315836 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 146/287] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:26 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:26.322048 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f94f28-6790-4c02-8541-605062853a2f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:26.464510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f94f28-6790-4c02-8541-605062853a2f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:26.497012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:26.516042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dfec8-3398-4c18-87ce-e7a95414cecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:26.516753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dfec8-3398-4c18-87ce-e7a95414cecc None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:26.517029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dfec8-3398-4c18-87ce-e7a95414cecc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:26.517295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dfec8-3398-4c18-87ce-e7a95414cecc None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa/users/8f7ad21310fa43fbbb36e1ca299cf471/roles/712114868a39430b80ec9c19cb91c947` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:26.519231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788dfec8-3398-4c18-87ce-e7a95414cecc None admin] RBAC: Authorizing `identity:create_grant(project_id=117d6fb504534ef2bd643528f241dffa, user_id=8f7ad21310fa43fbbb36e1ca299cf471, role_id=712114868a39430b80ec9c19cb91c947)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:26.520699 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788dfec8-3398-4c18-87ce-e7a95414cecc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:26.543361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 142/288] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:04:26 2022] PUT /identity/v3/projects/117d6fb504534ef2bd643528f241dffa/users/8f7ad21310fa43fbbb36e1ca299cf471/roles/712114868a39430b80ec9c19cb91c947 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:26.581030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f423e6e-a8af-4ca1-bbf5-34f83d452146 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:26.704623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:26.716843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f423e6e-a8af-4ca1-bbf5-34f83d452146 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:26.785367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320aec96-4023-457b-9f11-fd977b642a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:26.786856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aec96-4023-457b-9f11-fd977b642a79 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:26.787614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aec96-4023-457b-9f11-fd977b642a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:26.788344 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320aec96-4023-457b-9f11-fd977b642a79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:26.813829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320aec96-4023-457b-9f11-fd977b642a79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-320aec96-4023-457b-9f11-fd977b642a79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:26.819211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:26.820768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:26.822375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:26.822375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:26.822375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:26.822375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:26.822375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:26.822375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:26.822375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:26.822375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:26.822375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:26.822375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:26.822375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:26.822375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:26.822375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 147/289] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:26.830057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dfec8-3398-4c18-87ce-e7a95414cecc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:26.949625 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dfec8-3398-4c18-87ce-e7a95414cecc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:27.000308 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb000905-9d4d-4388-90e8-51ff9e05bd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:27.001051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb000905-9d4d-4388-90e8-51ff9e05bd23 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:27.001328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb000905-9d4d-4388-90e8-51ff9e05bd23 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:27.001598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb000905-9d4d-4388-90e8-51ff9e05bd23 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:27.002284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb000905-9d4d-4388-90e8-51ff9e05bd23 None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:27.002582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb000905-9d4d-4388-90e8-51ff9e05bd23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:27.003955 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb000905-9d4d-4388-90e8-51ff9e05bd23 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:27.013486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 143/290] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:27.019565 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320aec96-4023-457b-9f11-fd977b642a79 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:27.185794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320aec96-4023-457b-9f11-fd977b642a79 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:27.232722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0e076-7569-4998-8ce5-3955fc665876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:27.233681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0e076-7569-4998-8ce5-3955fc665876 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:27.233948 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0e076-7569-4998-8ce5-3955fc665876 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:27.234210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0e076-7569-4998-8ce5-3955fc665876 None admin] PATH_INFO: `/v3/users/ca0ac49016c2446fa93a38c25a584d64` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:27.235360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0e076-7569-4998-8ce5-3955fc665876 None admin] RBAC: Authorizing `identity:get_user(user_id=ca0ac49016c2446fa93a38c25a584d64)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:27.236909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0e076-7569-4998-8ce5-3955fc665876 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:27.238863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 148/291] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:27 2022] GET /identity/v3/users/ca0ac49016c2446fa93a38c25a584d64 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:27.244652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb000905-9d4d-4388-90e8-51ff9e05bd23 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:27.321823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:27.389755 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb000905-9d4d-4388-90e8-51ff9e05bd23 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:27.424581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17df64-12ec-4a25-a9f2-12d68cf6aa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:27.425258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17df64-12ec-4a25-a9f2-12d68cf6aa9a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:27.425538 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17df64-12ec-4a25-a9f2-12d68cf6aa9a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:27.425808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17df64-12ec-4a25-a9f2-12d68cf6aa9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:27.496165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17df64-12ec-4a25-a9f2-12d68cf6aa9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:27.497873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17df64-12ec-4a25-a9f2-12d68cf6aa9a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:27.539988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:27.599694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 144/292] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:27.609418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0e076-7569-4998-8ce5-3955fc665876 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:27.641600 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:27.754669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0e076-7569-4998-8ce5-3955fc665876 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:27.798550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f524059-17a2-4219-9bd7-6c14ed56670d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:27.799312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f524059-17a2-4219-9bd7-6c14ed56670d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:27.799585 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f524059-17a2-4219-9bd7-6c14ed56670d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:27.799847 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f524059-17a2-4219-9bd7-6c14ed56670d None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:27.801521 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f524059-17a2-4219-9bd7-6c14ed56670d None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:27.802950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f524059-17a2-4219-9bd7-6c14ed56670d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:27.804745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 149/293] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:27 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:27.810834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17df64-12ec-4a25-a9f2-12d68cf6aa9a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:27.955861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17df64-12ec-4a25-a9f2-12d68cf6aa9a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:28.001878 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861c400-9a21-4b7e-b04b-6fa2d21b4502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:28.002485 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861c400-9a21-4b7e-b04b-6fa2d21b4502 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:28.002758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861c400-9a21-4b7e-b04b-6fa2d21b4502 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:28.003027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861c400-9a21-4b7e-b04b-6fa2d21b4502 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:28.003764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861c400-9a21-4b7e-b04b-6fa2d21b4502 None admin] RBAC: Adding query filter params (role.id=2921a7146fd9408396ca98a4be4ecd3f, scope.project.id=117d6fb504534ef2bd643528f241dffa, user.id=ca0ac49016c2446fa93a38c25a584d64) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:28.004068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861c400-9a21-4b7e-b04b-6fa2d21b4502 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:28.005513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861c400-9a21-4b7e-b04b-6fa2d21b4502 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:28.016150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 145/294] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:27 2022] GET /identity/v3/role_assignments?user.id=ca0ac49016c2446fa93a38c25a584d64&scope.project.id=117d6fb504534ef2bd643528f241dffa&role.id=2921a7146fd9408396ca98a4be4ecd3f => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:28.604848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58398c3-4b40-42b0-827b-af74b5722f9a None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:28.605449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58398c3-4b40-42b0-827b-af74b5722f9a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:28.605449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58398c3-4b40-42b0-827b-af74b5722f9a None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:28.607087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 150/295] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:28.613000 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765507e2-a2d4-438b-88be-439bd4033554 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:28.613281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765507e2-a2d4-438b-88be-439bd4033554 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:28.613549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765507e2-a2d4-438b-88be-439bd4033554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:28.630543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:28.636437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-765507e2-a2d4-438b-88be-439bd4033554 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:28.637680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-765507e2-a2d4-438b-88be-439bd4033554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:28.717176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 146/296] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:28.827752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160127d-ebdb-453d-b2f9-58e3acfff227 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:28.828107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160127d-ebdb-453d-b2f9-58e3acfff227 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:28.828302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160127d-ebdb-453d-b2f9-58e3acfff227 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:28.830028 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 151/297] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:28.835202 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb7d2a-bdb7-4b0b-a78d-f4b4346fb515 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:28.835483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb7d2a-bdb7-4b0b-a78d-f4b4346fb515 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:28.835751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb7d2a-bdb7-4b0b-a78d-f4b4346fb515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:28.849502 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a6d31-8d35-4317-a6bc-a54bbbb8b4c1 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:28.850114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a6d31-8d35-4317-a6bc-a54bbbb8b4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:28.850572 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a6d31-8d35-4317-a6bc-a54bbbb8b4c1 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:28.852368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 152/298] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:04:28.855272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a1fb7d2a-bdb7-4b0b-a78d-f4b4346fb515 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:28.856474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a1fb7d2a-bdb7-4b0b-a78d-f4b4346fb515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:28.859136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7408dc-e534-4d49-aa41-1fbbec1074c8 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:28.859613 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7408dc-e534-4d49-aa41-1fbbec1074c8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:28.860062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7408dc-e534-4d49-aa41-1fbbec1074c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:28.879324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ae7408dc-e534-4d49-aa41-1fbbec1074c8 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:28.880945 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ae7408dc-e534-4d49-aa41-1fbbec1074c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:28.930474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 147/299] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:28.951238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 153/300] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:29.007764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb7d2a-bdb7-4b0b-a78d-f4b4346fb515 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:29.075658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:29.117677 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb7d2a-bdb7-4b0b-a78d-f4b4346fb515 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:29.156822 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea257a-bf94-4ed0-b82a-b8403847f8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:29.157833 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea257a-bf94-4ed0-b82a-b8403847f8df None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:29.158298 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea257a-bf94-4ed0-b82a-b8403847f8df None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:29.158787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea257a-bf94-4ed0-b82a-b8403847f8df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:29.169799 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea257a-bf94-4ed0-b82a-b8403847f8df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-00ea257a-bf94-4ed0-b82a-b8403847f8df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:29.174980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:29.175861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:29.176664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:29.176664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:29.176664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:29.176664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:29.176664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:29.176664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:29.176664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:29.176664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:29.176664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:29.176664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:29.176664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:29.176664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:29.179299 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 148/301] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:29.185771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7408dc-e534-4d49-aa41-1fbbec1074c8 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:29.201311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea257a-bf94-4ed0-b82a-b8403847f8df None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:29.205070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:29.296409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7408dc-e534-4d49-aa41-1fbbec1074c8 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:29.304382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea257a-bf94-4ed0-b82a-b8403847f8df None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:29.330059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:29.330707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a2931-1999-4758-94f4-62dad86b5832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:29.331356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a2931-1999-4758-94f4-62dad86b5832 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:29.331893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a2931-1999-4758-94f4-62dad86b5832 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:29.332215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a2931-1999-4758-94f4-62dad86b5832 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:29.333163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a2931-1999-4758-94f4-62dad86b5832 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:29.333441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a2931-1999-4758-94f4-62dad86b5832 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:29.334814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a2931-1999-4758-94f4-62dad86b5832 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:29.339416 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec04a2-9eb3-433d-94b5-ae25bc159f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:29.340046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec04a2-9eb3-433d-94b5-ae25bc159f73 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:29.340317 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec04a2-9eb3-433d-94b5-ae25bc159f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:29.340605 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec04a2-9eb3-433d-94b5-ae25bc159f73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:29.343480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 154/302] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:29.344501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c560ae-68c5-4ba4-acbf-c9e3280fab85 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:29.344801 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c560ae-68c5-4ba4-acbf-c9e3280fab85 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:29.345065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c560ae-68c5-4ba4-acbf-c9e3280fab85 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:29.348653 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 155/303] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:29.348653 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c560ae-68c5-4ba4-acbf-c9e3280fab85 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:29.354499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec04a2-9eb3-433d-94b5-ae25bc159f73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-b8ec04a2-9eb3-433d-94b5-ae25bc159f73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:29.359730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:29.360646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 149/304] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:04:29.361590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec04a2-9eb3-433d-94b5-ae25bc159f73 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:29.432679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:29.459817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c560ae-68c5-4ba4-acbf-c9e3280fab85 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:29.461621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec04a2-9eb3-433d-94b5-ae25bc159f73 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:29.498489 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c11ae-3223-4a74-aa65-e6c87b5533c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:29.499266 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c11ae-3223-4a74-aa65-e6c87b5533c9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:29.499592 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c11ae-3223-4a74-aa65-e6c87b5533c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:29.499907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c11ae-3223-4a74-aa65-e6c87b5533c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:29.501255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c11ae-3223-4a74-aa65-e6c87b5533c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:29.501599 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688af58-7536-4767-b890-ad488d861631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:29.502251 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688af58-7536-4767-b890-ad488d861631 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:29.502523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688af58-7536-4767-b890-ad488d861631 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:29.502787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688af58-7536-4767-b890-ad488d861631 None admin] PATH_INFO: `/v3/users/ba3276b8775b4117b3b8b13d06f76240` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:29.502886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c11ae-3223-4a74-aa65-e6c87b5533c9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:29.504942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 156/305] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:29.506146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e3179-9983-47d2-9d37-a3ce31fccdd2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:29.506497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e3179-9983-47d2-9d37-a3ce31fccdd2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:29.506760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e3179-9983-47d2-9d37-a3ce31fccdd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:29.524865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688af58-7536-4767-b890-ad488d861631 None admin] RBAC: Authorizing `identity:get_user(user_id=ba3276b8775b4117b3b8b13d06f76240)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:29.526906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688af58-7536-4767-b890-ad488d861631 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:29.527157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f34e3179-9983-47d2-9d37-a3ce31fccdd2 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:29.528474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f34e3179-9983-47d2-9d37-a3ce31fccdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:29.528956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 150/306] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:29 2022] GET /identity/v3/users/ba3276b8775b4117b3b8b13d06f76240 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:29.529875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688af58-7536-4767-b890-ad488d861631 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:29.588968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:29.609714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 157/307] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:29.610559 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e3179-9983-47d2-9d37-a3ce31fccdd2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:29.653884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688af58-7536-4767-b890-ad488d861631 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:29.696907 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce48720-7e07-4744-a629-8cfab6f033ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:29.697703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce48720-7e07-4744-a629-8cfab6f033ef None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:29.698049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce48720-7e07-4744-a629-8cfab6f033ef None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:29.698406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce48720-7e07-4744-a629-8cfab6f033ef None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:29.699499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce48720-7e07-4744-a629-8cfab6f033ef None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:29.699724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce48720-7e07-4744-a629-8cfab6f033ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:29.701571 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce48720-7e07-4744-a629-8cfab6f033ef None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:29.704994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:29.713304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 151/308] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:29.714309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce48720-7e07-4744-a629-8cfab6f033ef None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:29.737940 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e3179-9983-47d2-9d37-a3ce31fccdd2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:29.776029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f872e7b-6b59-425e-a89a-3ff216494e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:29.776670 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f872e7b-6b59-425e-a89a-3ff216494e60 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:29.776947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f872e7b-6b59-425e-a89a-3ff216494e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:29.777211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f872e7b-6b59-425e-a89a-3ff216494e60 None admin] PATH_INFO: `/v3/groups` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:29.777857 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f872e7b-6b59-425e-a89a-3ff216494e60 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:29.779290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f872e7b-6b59-425e-a89a-3ff216494e60 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:29.797271 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 158/309] 10.222.0.75 () {60 vars in 1134 bytes} [Fri Sep 23 15:04:29 2022] POST /identity/v3/groups => generated 229 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:04:29.798045 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f872e7b-6b59-425e-a89a-3ff216494e60 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:29.857063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce48720-7e07-4744-a629-8cfab6f033ef None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:29.867128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:29.915204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a617787-e772-4021-aeb5-2781da076596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:29.916481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a617787-e772-4021-aeb5-2781da076596 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:29.917184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a617787-e772-4021-aeb5-2781da076596 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:29.917450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a617787-e772-4021-aeb5-2781da076596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:29.919002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a617787-e772-4021-aeb5-2781da076596 None admin] RBAC: Adding query filter params (role.id=712114868a39430b80ec9c19cb91c947, user.id=ba3276b8775b4117b3b8b13d06f76240, scope.system=all) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:29.920783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a617787-e772-4021-aeb5-2781da076596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:29.923447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a617787-e772-4021-aeb5-2781da076596 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:29.932514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 152/310] 10.222.0.75 () {56 vars in 1308 bytes} [Fri Sep 23 15:04:29 2022] GET /identity/v3/role_assignments?user.id=ba3276b8775b4117b3b8b13d06f76240&scope.system=all&role.id=712114868a39430b80ec9c19cb91c947 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:29.933831 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a617787-e772-4021-aeb5-2781da076596 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:29.984067 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f872e7b-6b59-425e-a89a-3ff216494e60 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:30.040957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec276530-3c32-42ea-afe0-604b226ace22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:30.041625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec276530-3c32-42ea-afe0-604b226ace22 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:30.041913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec276530-3c32-42ea-afe0-604b226ace22 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:30.042298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec276530-3c32-42ea-afe0-604b226ace22 None admin] PATH_INFO: `/v3/users/cd002ecfa3684ab0b1500496760db640` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:30.045022 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec276530-3c32-42ea-afe0-604b226ace22 None admin] RBAC: Authorizing `identity:get_user(user_id=cd002ecfa3684ab0b1500496760db640)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:30.046062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec276530-3c32-42ea-afe0-604b226ace22 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:30.047809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 159/311] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:29 2022] GET /identity/v3/users/cd002ecfa3684ab0b1500496760db640 => generated 293 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:30.056114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec276530-3c32-42ea-afe0-604b226ace22 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:30.062052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:30.089647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a617787-e772-4021-aeb5-2781da076596 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:30.143743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af12617-470c-42d4-9ca9-44efe92620e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:30.144951 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af12617-470c-42d4-9ca9-44efe92620e2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:30.145424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af12617-470c-42d4-9ca9-44efe92620e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:30.146037 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af12617-470c-42d4-9ca9-44efe92620e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:30.161057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af12617-470c-42d4-9ca9-44efe92620e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-6af12617-470c-42d4-9ca9-44efe92620e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:30.167278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:30.168166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:30.168990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:30.168990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:30.168990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:30.168990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:30.168990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:30.168990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:30.168990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:30.168990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:30.168990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:30.168990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:30.168990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:30.168990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:30.177964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 153/312] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:30.180028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af12617-470c-42d4-9ca9-44efe92620e2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:30.188545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec276530-3c32-42ea-afe0-604b226ace22 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:30.215660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:30.225919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107cc58-dd42-4d39-a018-e8b06290b147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:30.226587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107cc58-dd42-4d39-a018-e8b06290b147 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:30.226856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107cc58-dd42-4d39-a018-e8b06290b147 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:30.227121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107cc58-dd42-4d39-a018-e8b06290b147 None admin] PATH_INFO: `/v3/system/users/cd002ecfa3684ab0b1500496760db640/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:30.228781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107cc58-dd42-4d39-a018-e8b06290b147 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cd002ecfa3684ab0b1500496760db640, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:30.230167 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107cc58-dd42-4d39-a018-e8b06290b147 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:30.243278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 160/313] 10.222.0.75 () {58 vars in 1334 bytes} [Fri Sep 23 15:04:30 2022] PUT /identity/v3/system/users/cd002ecfa3684ab0b1500496760db640/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:30.310603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af12617-470c-42d4-9ca9-44efe92620e2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:30.352210 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed737f1-1051-4102-963d-df1c2c8b0201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:30.353118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed737f1-1051-4102-963d-df1c2c8b0201 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:30.353713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed737f1-1051-4102-963d-df1c2c8b0201 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:30.354281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed737f1-1051-4102-963d-df1c2c8b0201 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:30.355286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed737f1-1051-4102-963d-df1c2c8b0201 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:30.355740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed737f1-1051-4102-963d-df1c2c8b0201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:30.357333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed737f1-1051-4102-963d-df1c2c8b0201 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:30.372701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 154/314] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:30.374971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107cc58-dd42-4d39-a018-e8b06290b147 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:30.459988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:30.500956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107cc58-dd42-4d39-a018-e8b06290b147 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:30.544267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d960cd1-a37c-48ec-bbb8-0336af74efb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:30.544943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d960cd1-a37c-48ec-bbb8-0336af74efb1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:30.545351 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d960cd1-a37c-48ec-bbb8-0336af74efb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:30.545521 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d960cd1-a37c-48ec-bbb8-0336af74efb1 None admin] PATH_INFO: `/v3/users/8f7ad21310fa43fbbb36e1ca299cf471` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:30.546682 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d960cd1-a37c-48ec-bbb8-0336af74efb1 None admin] RBAC: Authorizing `identity:get_user(user_id=8f7ad21310fa43fbbb36e1ca299cf471)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:30.548128 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d960cd1-a37c-48ec-bbb8-0336af74efb1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:30.556132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 161/315] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:30 2022] GET /identity/v3/users/8f7ad21310fa43fbbb36e1ca299cf471 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:30.558192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed737f1-1051-4102-963d-df1c2c8b0201 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:30.655750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:30.665147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed737f1-1051-4102-963d-df1c2c8b0201 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:30.700786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a3187-27f7-4f0b-b1fb-a996629bc878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:30.701446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a3187-27f7-4f0b-b1fb-a996629bc878 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:30.701724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a3187-27f7-4f0b-b1fb-a996629bc878 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:30.701995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a3187-27f7-4f0b-b1fb-a996629bc878 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:30.773766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a3187-27f7-4f0b-b1fb-a996629bc878 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:30.775142 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a3187-27f7-4f0b-b1fb-a996629bc878 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:30.871636 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:30.888294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 155/316] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:30.894028 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d960cd1-a37c-48ec-bbb8-0336af74efb1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:31.005011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d960cd1-a37c-48ec-bbb8-0336af74efb1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:31.041425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a495623f-016c-4e9a-9f2d-00dcfd65ea69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:31.042445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495623f-016c-4e9a-9f2d-00dcfd65ea69 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:31.042928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495623f-016c-4e9a-9f2d-00dcfd65ea69 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:31.043380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495623f-016c-4e9a-9f2d-00dcfd65ea69 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:31.044750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495623f-016c-4e9a-9f2d-00dcfd65ea69 None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:31.046382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a495623f-016c-4e9a-9f2d-00dcfd65ea69 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:31.048657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 162/317] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:30 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:31.060397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a3187-27f7-4f0b-b1fb-a996629bc878 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:31.160871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a3187-27f7-4f0b-b1fb-a996629bc878 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:31.197659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502c99e-aeee-4c37-97a2-b78b88822fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:31.198265 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502c99e-aeee-4c37-97a2-b78b88822fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:31.198540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502c99e-aeee-4c37-97a2-b78b88822fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:31.198805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502c99e-aeee-4c37-97a2-b78b88822fc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:31.199529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502c99e-aeee-4c37-97a2-b78b88822fc2 None admin] RBAC: Adding query filter params (role.id=712114868a39430b80ec9c19cb91c947, scope.project.id=117d6fb504534ef2bd643528f241dffa, user.id=8f7ad21310fa43fbbb36e1ca299cf471) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:31.199838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502c99e-aeee-4c37-97a2-b78b88822fc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:31.201285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502c99e-aeee-4c37-97a2-b78b88822fc2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:31.207780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 156/318] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:04:31 2022] GET /identity/v3/role_assignments?user.id=8f7ad21310fa43fbbb36e1ca299cf471&scope.project.id=117d6fb504534ef2bd643528f241dffa&role.id=712114868a39430b80ec9c19cb91c947 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:32.182665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65a47f-9abf-4735-90a4-ead9f7040e1b None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:32.183252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65a47f-9abf-4735-90a4-ead9f7040e1b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:32.183252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65a47f-9abf-4735-90a4-ead9f7040e1b None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:32.184777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 163/319] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:32.189940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f455d16-9336-4e27-9018-ccba99404349 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:32.190359 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f455d16-9336-4e27-9018-ccba99404349 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:32.190429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f455d16-9336-4e27-9018-ccba99404349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:32.210366 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a9210-f79a-4ced-9813-d0277ec7c966 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:32.210663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0f455d16-9336-4e27-9018-ccba99404349 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:32.210764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a9210-f79a-4ced-9813-d0277ec7c966 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:32.210955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a9210-f79a-4ced-9813-d0277ec7c966 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:32.211745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0f455d16-9336-4e27-9018-ccba99404349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:32.212491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 164/320] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:04:32.217840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2ac23-75ce-4238-98fa-b11ba7dded11 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:32.218116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2ac23-75ce-4238-98fa-b11ba7dded11 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:32.218388 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2ac23-75ce-4238-98fa-b11ba7dded11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:32.225596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:32.237575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-76b2ac23-75ce-4238-98fa-b11ba7dded11 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:32.239023 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-76b2ac23-75ce-4238-98fa-b11ba7dded11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:32.280839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:32.308617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 157/321] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:32.338334 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 165/322] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:32.508331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f06b5a-3791-4f8a-a6a2-83ec19b0be04 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:32.508530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f06b5a-3791-4f8a-a6a2-83ec19b0be04 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:32.508821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f06b5a-3791-4f8a-a6a2-83ec19b0be04 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:32.510522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 158/323] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:32.515844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e920bfa-69a8-4e00-aed3-82d7caa15117 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:32.516085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e920bfa-69a8-4e00-aed3-82d7caa15117 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:32.516384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e920bfa-69a8-4e00-aed3-82d7caa15117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:32.536651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2e920bfa-69a8-4e00-aed3-82d7caa15117 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:32.537830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2e920bfa-69a8-4e00-aed3-82d7caa15117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:32.570877 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f06b5a-3791-4f8a-a6a2-83ec19b0be04 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:32.608848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 166/324] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:32.614935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e920bfa-69a8-4e00-aed3-82d7caa15117 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:32.686683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f06b5a-3791-4f8a-a6a2-83ec19b0be04 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:32.686926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:32.725158 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9446c83-5521-40bd-831b-88cac2b1b86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:32.726076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9446c83-5521-40bd-831b-88cac2b1b86f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:32.726332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9446c83-5521-40bd-831b-88cac2b1b86f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:32.726609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9446c83-5521-40bd-831b-88cac2b1b86f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:32.727757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9446c83-5521-40bd-831b-88cac2b1b86f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:32.727909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e920bfa-69a8-4e00-aed3-82d7caa15117 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:32.729450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9446c83-5521-40bd-831b-88cac2b1b86f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:32.731258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 159/325] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:32.736808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9446c83-5521-40bd-831b-88cac2b1b86f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:32.766795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b49998-2f02-436c-99ed-4bb0811d3837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:32.767349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b49998-2f02-436c-99ed-4bb0811d3837 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:32.767621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b49998-2f02-436c-99ed-4bb0811d3837 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:32.767894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b49998-2f02-436c-99ed-4bb0811d3837 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:32.778716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b49998-2f02-436c-99ed-4bb0811d3837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-95b49998-2f02-436c-99ed-4bb0811d3837 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:32.783320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:32.784387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:32.785402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:32.785402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:32.785402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:32.785402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:32.785402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:32.785402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:32.785402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:32.785402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:32.785402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:32.785402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:32.785402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:32.785402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:32.785402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 167/326] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:04:32.792310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b49998-2f02-436c-99ed-4bb0811d3837 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:32.800922 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:32.850211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9446c83-5521-40bd-831b-88cac2b1b86f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:32.889704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f6da6-3d34-4f44-959c-964b4b9bd075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:32.890301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f6da6-3d34-4f44-959c-964b4b9bd075 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:32.890566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f6da6-3d34-4f44-959c-964b4b9bd075 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:32.890830 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f6da6-3d34-4f44-959c-964b4b9bd075 None admin] PATH_INFO: `/v3/groups` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:32.891461 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f6da6-3d34-4f44-959c-964b4b9bd075 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:32.892931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f6da6-3d34-4f44-959c-964b4b9bd075 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:32.896823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b49998-2f02-436c-99ed-4bb0811d3837 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:32.910507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 160/327] 10.222.0.75 () {60 vars in 1134 bytes} [Fri Sep 23 15:04:32 2022] POST /identity/v3/groups => generated 226 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:04:32.917848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f6da6-3d34-4f44-959c-964b4b9bd075 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:32.921748 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:32.935838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb9e08-a62a-4b8d-8c61-e1fa99f01712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:32.936459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb9e08-a62a-4b8d-8c61-e1fa99f01712 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:32.936774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb9e08-a62a-4b8d-8c61-e1fa99f01712 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:32.937050 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb9e08-a62a-4b8d-8c61-e1fa99f01712 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:32.937741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb9e08-a62a-4b8d-8c61-e1fa99f01712 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:32.938044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb9e08-a62a-4b8d-8c61-e1fa99f01712 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:32.939477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb9e08-a62a-4b8d-8c61-e1fa99f01712 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:32.949235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 168/328] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:32.955013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb9e08-a62a-4b8d-8c61-e1fa99f01712 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:33.031532 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f6da6-3d34-4f44-959c-964b4b9bd075 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:33.050299 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:33.070404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb9e08-a62a-4b8d-8c61-e1fa99f01712 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:33.081483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf15b9-7009-466a-ae62-8b6fca781f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:33.082466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf15b9-7009-466a-ae62-8b6fca781f30 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:33.082736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf15b9-7009-466a-ae62-8b6fca781f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:33.083001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf15b9-7009-466a-ae62-8b6fca781f30 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:33.100635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf15b9-7009-466a-ae62-8b6fca781f30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-d9bf15b9-7009-466a-ae62-8b6fca781f30 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:33.105956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:33.107600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:33.108750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:33.108750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:33.108750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:33.108750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:33.108750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:33.108750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:33.108750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:33.108750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:33.108750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:33.108750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:33.108750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:33.108750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:33.108750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 161/329] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:33.114031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf15b9-7009-466a-ae62-8b6fca781f30 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:33.122766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b92ddd-e3df-4a7d-8510-92272f5bd71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:33.123564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b92ddd-e3df-4a7d-8510-92272f5bd71c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:33.123671 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b92ddd-e3df-4a7d-8510-92272f5bd71c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:33.123923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b92ddd-e3df-4a7d-8510-92272f5bd71c None admin] PATH_INFO: `/v3/users/ba3276b8775b4117b3b8b13d06f76240` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:33.125146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b92ddd-e3df-4a7d-8510-92272f5bd71c None admin] RBAC: Authorizing `identity:get_user(user_id=ba3276b8775b4117b3b8b13d06f76240)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:33.126601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b92ddd-e3df-4a7d-8510-92272f5bd71c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:33.128362 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 169/330] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:32 2022] GET /identity/v3/users/ba3276b8775b4117b3b8b13d06f76240 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:33.133858 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b92ddd-e3df-4a7d-8510-92272f5bd71c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:33.202223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:33.245243 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf15b9-7009-466a-ae62-8b6fca781f30 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:33.254790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b92ddd-e3df-4a7d-8510-92272f5bd71c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:33.293356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2195ab-3240-4ab5-ad0e-f7ead807b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:33.293904 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2195ab-3240-4ab5-ad0e-f7ead807b95d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:33.295271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2195ab-3240-4ab5-ad0e-f7ead807b95d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:33.295554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2195ab-3240-4ab5-ad0e-f7ead807b95d None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:33.296377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2195ab-3240-4ab5-ad0e-f7ead807b95d None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:33.296739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2195ab-3240-4ab5-ad0e-f7ead807b95d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:33.298294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2195ab-3240-4ab5-ad0e-f7ead807b95d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:33.303217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9d9cb-0488-43e5-b105-404bff253885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:33.303880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9d9cb-0488-43e5-b105-404bff253885 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:33.304253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9d9cb-0488-43e5-b105-404bff253885 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:33.304524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9d9cb-0488-43e5-b105-404bff253885 None admin] PATH_INFO: `/v3/system/users/ba3276b8775b4117b3b8b13d06f76240/roles/712114868a39430b80ec9c19cb91c947` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:33.305971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9d9cb-0488-43e5-b105-404bff253885 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ba3276b8775b4117b3b8b13d06f76240, role_id=712114868a39430b80ec9c19cb91c947)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:33.307538 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9d9cb-0488-43e5-b105-404bff253885 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:33.309731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 162/331] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:33.315299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 170/332] 10.222.0.75 () {58 vars in 1334 bytes} [Fri Sep 23 15:04:33 2022] PUT /identity/v3/system/users/ba3276b8775b4117b3b8b13d06f76240/roles/712114868a39430b80ec9c19cb91c947 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:33.316027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2195ab-3240-4ab5-ad0e-f7ead807b95d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:33.361327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:33.442436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2195ab-3240-4ab5-ad0e-f7ead807b95d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:33.482418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49fa9e1-8a01-4561-b74a-ce028b490862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:33.483438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fa9e1-8a01-4561-b74a-ce028b490862 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:33.483709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fa9e1-8a01-4561-b74a-ce028b490862 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:33.483972 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49fa9e1-8a01-4561-b74a-ce028b490862 None admin] PATH_INFO: `/v3/users/cd002ecfa3684ab0b1500496760db640` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:33.485217 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fa9e1-8a01-4561-b74a-ce028b490862 None admin] RBAC: Authorizing `identity:get_user(user_id=cd002ecfa3684ab0b1500496760db640)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:33.486805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49fa9e1-8a01-4561-b74a-ce028b490862 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:33.488809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 163/333] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:33 2022] GET /identity/v3/users/cd002ecfa3684ab0b1500496760db640 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:33.494699 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9d9cb-0488-43e5-b105-404bff253885 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:33.618495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9d9cb-0488-43e5-b105-404bff253885 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:33.637324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:33.656957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3735bd-026e-449d-9cc7-5bbbda802550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:33.657569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3735bd-026e-449d-9cc7-5bbbda802550 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:33.657942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3735bd-026e-449d-9cc7-5bbbda802550 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:33.658276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3735bd-026e-449d-9cc7-5bbbda802550 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:33.659021 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3735bd-026e-449d-9cc7-5bbbda802550 None admin] RBAC: Adding query filter params (role.id=2921a7146fd9408396ca98a4be4ecd3f, user.id=cd002ecfa3684ab0b1500496760db640, scope.system=all) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:33.659330 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3735bd-026e-449d-9cc7-5bbbda802550 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:33.660825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3735bd-026e-449d-9cc7-5bbbda802550 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:33.680741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 171/334] 10.222.0.75 () {56 vars in 1308 bytes} [Fri Sep 23 15:04:33 2022] GET /identity/v3/role_assignments?user.id=cd002ecfa3684ab0b1500496760db640&scope.system=all&role.id=2921a7146fd9408396ca98a4be4ecd3f => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:35.372087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d7703-28c4-4499-936f-21eead4cda02 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:35.373117 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d7703-28c4-4499-936f-21eead4cda02 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:35.373618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d7703-28c4-4499-936f-21eead4cda02 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:35.375189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 164/335] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:35.380917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4de4db-14b3-4dd6-8f16-5a9b9fa34ae9 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:35.381364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4de4db-14b3-4dd6-8f16-5a9b9fa34ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:35.381561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4de4db-14b3-4dd6-8f16-5a9b9fa34ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:35.405021 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-de4de4db-14b3-4dd6-8f16-5a9b9fa34ae9 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:35.406275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-de4de4db-14b3-4dd6-8f16-5a9b9fa34ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:35.507313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 172/336] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:35.599971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925107da-bce1-4ae0-99da-8064678ec513 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:35.600297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925107da-bce1-4ae0-99da-8064678ec513 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:35.600508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925107da-bce1-4ae0-99da-8064678ec513 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:35.602153 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 165/337] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:35.607569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b7435-3f7b-4dcb-b804-150ce019b1fb None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:35.607855 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b7435-3f7b-4dcb-b804-150ce019b1fb None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:35.608449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b7435-3f7b-4dcb-b804-150ce019b1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:35.630005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a93b7435-3f7b-4dcb-b804-150ce019b1fb None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:35.631111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a93b7435-3f7b-4dcb-b804-150ce019b1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:35.726383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 173/338] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:35.982859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925107da-bce1-4ae0-99da-8064678ec513 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:36.114207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925107da-bce1-4ae0-99da-8064678ec513 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:36.116835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:36.160133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58795c-454b-4c67-8f16-19dc485b9604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:36.160803 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58795c-454b-4c67-8f16-19dc485b9604 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:36.161083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58795c-454b-4c67-8f16-19dc485b9604 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:36.161346 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58795c-454b-4c67-8f16-19dc485b9604 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:36.179247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58795c-454b-4c67-8f16-19dc485b9604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-fb58795c-454b-4c67-8f16-19dc485b9604 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:36.184449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:36.185441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:36.186241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:36.186241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:36.186241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:36.186241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:36.186241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:36.186241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:36.186241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:36.186241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:36.186241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:36.186241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:36.186241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:36.186241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:36.186241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 166/339] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:36.191679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b7435-3f7b-4dcb-b804-150ce019b1fb None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:36.293740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:36.303715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfc434-6013-44db-82d2-09ad745d7e0e None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:36.304022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfc434-6013-44db-82d2-09ad745d7e0e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:36.304257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfc434-6013-44db-82d2-09ad745d7e0e None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:36.305892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 167/340] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:36.311601 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c94d67-176d-434d-8d21-3e6ae9ffd409 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:36.311876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c94d67-176d-434d-8d21-3e6ae9ffd409 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:36.312146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c94d67-176d-434d-8d21-3e6ae9ffd409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:36.351567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-18c94d67-176d-434d-8d21-3e6ae9ffd409 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:36.357123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-18c94d67-176d-434d-8d21-3e6ae9ffd409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:36.435567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b7435-3f7b-4dcb-b804-150ce019b1fb None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:36.492852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 168/341] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:36.494873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c94d67-176d-434d-8d21-3e6ae9ffd409 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:36.506733 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a5a0c-5f21-4a2d-bba6-10d0ac1ddaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:36.507691 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a5a0c-5f21-4a2d-bba6-10d0ac1ddaa4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:36.508155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a5a0c-5f21-4a2d-bba6-10d0ac1ddaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:36.508672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a5a0c-5f21-4a2d-bba6-10d0ac1ddaa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:36.509638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a5a0c-5f21-4a2d-bba6-10d0ac1ddaa4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:36.510366 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a5a0c-5f21-4a2d-bba6-10d0ac1ddaa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:36.512010 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a5a0c-5f21-4a2d-bba6-10d0ac1ddaa4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:36.527403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 174/342] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:36.533679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a5a0c-5f21-4a2d-bba6-10d0ac1ddaa4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:36.651860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:36.654644 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c94d67-176d-434d-8d21-3e6ae9ffd409 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:36.719267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:36.723913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a5a0c-5f21-4a2d-bba6-10d0ac1ddaa4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:36.729318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899c172-72ab-40f9-96b0-2ba7bacca660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:36.729969 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899c172-72ab-40f9-96b0-2ba7bacca660 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:36.730242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899c172-72ab-40f9-96b0-2ba7bacca660 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:36.730510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899c172-72ab-40f9-96b0-2ba7bacca660 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:36.749851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899c172-72ab-40f9-96b0-2ba7bacca660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-e899c172-72ab-40f9-96b0-2ba7bacca660 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:36.760640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:36.761732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:36.762686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:36.762686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:36.762686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:36.762686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:36.762686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:36.762686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:36.762686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:36.762686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:36.762686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:36.762686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:36.762686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:36.762686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:36.764848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 169/343] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:04:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:36.764960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83309255-bb13-438b-8844-f385fc4b00fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:36.765577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83309255-bb13-438b-8844-f385fc4b00fb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:36.766033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83309255-bb13-438b-8844-f385fc4b00fb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:36.766435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83309255-bb13-438b-8844-f385fc4b00fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:36.766746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dc30d-f4f0-4678-9e87-7e1772f84dc5 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:36.767022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dc30d-f4f0-4678-9e87-7e1772f84dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:36.767290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dc30d-f4f0-4678-9e87-7e1772f84dc5 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:36.772938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 170/344] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:36.773710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dc30d-f4f0-4678-9e87-7e1772f84dc5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:36.852237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83309255-bb13-438b-8844-f385fc4b00fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:36.854032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83309255-bb13-438b-8844-f385fc4b00fb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:36.962571 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dc30d-f4f0-4678-9e87-7e1772f84dc5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:36.967885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:36.973906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 175/345] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 15:04:36.975883 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a42f1-8382-4695-a3b0-75caec152274 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:36.976630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a42f1-8382-4695-a3b0-75caec152274 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:36.977044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a42f1-8382-4695-a3b0-75caec152274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:37.000451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-172a42f1-8382-4695-a3b0-75caec152274 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:37.002056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-172a42f1-8382-4695-a3b0-75caec152274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:37.002283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beddae0-5e97-407b-abe2-8ef6969876cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:37.003069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beddae0-5e97-407b-abe2-8ef6969876cd None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:37.003483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beddae0-5e97-407b-abe2-8ef6969876cd None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:37.003889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beddae0-5e97-407b-abe2-8ef6969876cd None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:37.005861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beddae0-5e97-407b-abe2-8ef6969876cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:37.006348 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beddae0-5e97-407b-abe2-8ef6969876cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:37.007931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beddae0-5e97-407b-abe2-8ef6969876cd None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:37.019686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 171/346] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:04:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:37.020867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beddae0-5e97-407b-abe2-8ef6969876cd None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:37.042993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:37.093531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 176/347] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:37.094614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a42f1-8382-4695-a3b0-75caec152274 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:37.178831 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beddae0-5e97-407b-abe2-8ef6969876cd None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:37.244429 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a42f1-8382-4695-a3b0-75caec152274 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:37.267714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4828e9-0cf3-47f1-91a6-f6686926eab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:37.268451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4828e9-0cf3-47f1-91a6-f6686926eab5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:37.268946 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4828e9-0cf3-47f1-91a6-f6686926eab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:37.269390 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4828e9-0cf3-47f1-91a6-f6686926eab5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:37.280991 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:37.285289 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668a330-efe2-426b-9e8a-33d15966e9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:37.286215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668a330-efe2-426b-9e8a-33d15966e9f8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:37.286662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668a330-efe2-426b-9e8a-33d15966e9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:37.287289 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668a330-efe2-426b-9e8a-33d15966e9f8 None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:37.290966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668a330-efe2-426b-9e8a-33d15966e9f8 None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:37.291104 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4828e9-0cf3-47f1-91a6-f6686926eab5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:37.293156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668a330-efe2-426b-9e8a-33d15966e9f8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:37.295345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 177/348] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:37 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:37.296267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668a330-efe2-426b-9e8a-33d15966e9f8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-aa4828e9-0cf3-47f1-91a6-f6686926eab5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:37.299781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:37.300703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:37.301519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:37.301519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:37.301519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:37.301519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:37.301519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:37.301519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:37.301519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:37.301519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:37.301519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:37.301519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:37.301519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 15:04:37.301519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:37.302129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 172/349] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:37.303353 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4828e9-0cf3-47f1-91a6-f6686926eab5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:37.460306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:37.499920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668a330-efe2-426b-9e8a-33d15966e9f8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:37.508035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4828e9-0cf3-47f1-91a6-f6686926eab5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:37.564339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecde880-7f30-4ce9-bd21-6c48256e8e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:37.564997 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecde880-7f30-4ce9-bd21-6c48256e8e80 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:37.565266 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecde880-7f30-4ce9-bd21-6c48256e8e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:37.565530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecde880-7f30-4ce9-bd21-6c48256e8e80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:37.574390 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a1c39-e7ff-4c1b-92f0-de161a2e97f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:37.575463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a1c39-e7ff-4c1b-92f0-de161a2e97f8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:37.575947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a1c39-e7ff-4c1b-92f0-de161a2e97f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:37.576484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a1c39-e7ff-4c1b-92f0-de161a2e97f8 None admin] PATH_INFO: `/v3/groups/362a436695ce418b9853d511fd6b8243` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:37.584051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a1c39-e7ff-4c1b-92f0-de161a2e97f8 None admin] RBAC: Authorizing `identity:get_group(group_id=362a436695ce418b9853d511fd6b8243)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:37.586050 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a1c39-e7ff-4c1b-92f0-de161a2e97f8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:37.588040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 173/350] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:37 2022] GET /identity/v3/groups/362a436695ce418b9853d511fd6b8243 => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:37.589244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a1c39-e7ff-4c1b-92f0-de161a2e97f8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:37.645550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:37.657048 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecde880-7f30-4ce9-bd21-6c48256e8e80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:37.658934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecde880-7f30-4ce9-bd21-6c48256e8e80 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:37.732559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a1c39-e7ff-4c1b-92f0-de161a2e97f8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:37.776683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 178/351] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:37.777646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecde880-7f30-4ce9-bd21-6c48256e8e80 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:37.792694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ba418-f4ec-4ea5-af80-639b13dbdea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:37.793545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ba418-f4ec-4ea5-af80-639b13dbdea2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:37.793949 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ba418-f4ec-4ea5-af80-639b13dbdea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:37.794354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ba418-f4ec-4ea5-af80-639b13dbdea2 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:37.795190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ba418-f4ec-4ea5-af80-639b13dbdea2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:37.795628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ba418-f4ec-4ea5-af80-639b13dbdea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:37.797237 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ba418-f4ec-4ea5-af80-639b13dbdea2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:37.807091 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 174/352] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:37.808098 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ba418-f4ec-4ea5-af80-639b13dbdea2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:37.812068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:37.894165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:37.904897 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecde880-7f30-4ce9-bd21-6c48256e8e80 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:37.929457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ba418-f4ec-4ea5-af80-639b13dbdea2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:37.954936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d051a5-8c24-437d-bad2-30c013750485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:37.955775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d051a5-8c24-437d-bad2-30c013750485 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:37.956051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d051a5-8c24-437d-bad2-30c013750485 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:37.956317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d051a5-8c24-437d-bad2-30c013750485 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:37.971645 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d051a5-8c24-437d-bad2-30c013750485 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:37.975874 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0389593-d456-475b-aac7-7be48713739e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:37.976481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0389593-d456-475b-aac7-7be48713739e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:37.976810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0389593-d456-475b-aac7-7be48713739e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:37.977076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0389593-d456-475b-aac7-7be48713739e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:37.979750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0389593-d456-475b-aac7-7be48713739e None admin] RBAC: Adding query filter params (group.id=362a436695ce418b9853d511fd6b8243, role.id=2921a7146fd9408396ca98a4be4ecd3f, scope.project.id=9edb599ff8bb4c67b467b712ea2dc700) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d5d051a5-8c24-437d-bad2-30c013750485 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:37.979869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:37.980986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:37.981783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:37.981783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:37.981783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:37.981783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:37.981783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:37.981783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:37.981783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:37.981783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:37.981783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:37.981783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:37.981783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:37.981783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:37.982371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0389593-d456-475b-aac7-7be48713739e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:37.982439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 179/353] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:37.982439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238787e-c805-48e4-a483-5f451673ea52 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:37.982439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238787e-c805-48e4-a483-5f451673ea52 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:37.982439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238787e-c805-48e4-a483-5f451673ea52 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:37.983622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 180/354] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:37.983717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0389593-d456-475b-aac7-7be48713739e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:37.989424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238787e-c805-48e4-a483-5f451673ea52 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:37.991692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 175/355] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Sep 23 15:04:37 2022] GET /identity/v3/role_assignments?group.id=362a436695ce418b9853d511fd6b8243&scope.project.id=9edb599ff8bb4c67b467b712ea2dc700&role.id=2921a7146fd9408396ca98a4be4ecd3f => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:37.993381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0389593-d456-475b-aac7-7be48713739e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:38.092287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:38.132792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238787e-c805-48e4-a483-5f451673ea52 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:38.142873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0389593-d456-475b-aac7-7be48713739e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:38.175639 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd79b49-a99a-42ef-b805-34d026b33418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:38.176551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd79b49-a99a-42ef-b805-34d026b33418 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:38.176854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd79b49-a99a-42ef-b805-34d026b33418 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:38.177120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd79b49-a99a-42ef-b805-34d026b33418 None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:38.178834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd79b49-a99a-42ef-b805-34d026b33418 None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:38.180617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd79b49-a99a-42ef-b805-34d026b33418 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:38.182889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 181/356] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:37 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:38.184106 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4b7cc-23a0-4486-83c1-c1d133341a5e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:38.184380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4b7cc-23a0-4486-83c1-c1d133341a5e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:38.184678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4b7cc-23a0-4486-83c1-c1d133341a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:38.184929 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c1ae8-67bd-40df-9176-383be6943c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:38.185754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c1ae8-67bd-40df-9176-383be6943c83 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:38.186292 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c1ae8-67bd-40df-9176-383be6943c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:38.186731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c1ae8-67bd-40df-9176-383be6943c83 None admin] PATH_INFO: `/v3/users/ba3276b8775b4117b3b8b13d06f76240` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:38.188097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c1ae8-67bd-40df-9176-383be6943c83 None admin] RBAC: Authorizing `identity:get_user(user_id=ba3276b8775b4117b3b8b13d06f76240)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:38.190070 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c1ae8-67bd-40df-9176-383be6943c83 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:38.197076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 176/357] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:04:37 2022] GET /identity/v3/users/ba3276b8775b4117b3b8b13d06f76240 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:38.198404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c1ae8-67bd-40df-9176-383be6943c83 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:38.205327 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-eab4b7cc-23a0-4486-83c1-c1d133341a5e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:38.206514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-eab4b7cc-23a0-4486-83c1-c1d133341a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:38.228257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:38.293057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 182/358] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:38.294671 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffccb7-8324-40a6-90e7-6e803400cd2c None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:38.294671 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffccb7-8324-40a6-90e7-6e803400cd2c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:38.294939 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffccb7-8324-40a6-90e7-6e803400cd2c None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:38.296331 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 183/359] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:38.296903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffccb7-8324-40a6-90e7-6e803400cd2c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:38.342853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c1ae8-67bd-40df-9176-383be6943c83 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:38.389584 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecebcf-d977-44fc-a151-bb99ad68bb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:38.390604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecebcf-d977-44fc-a151-bb99ad68bb44 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:38.391097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecebcf-d977-44fc-a151-bb99ad68bb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:38.391550 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecebcf-d977-44fc-a151-bb99ad68bb44 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:38.392940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecebcf-d977-44fc-a151-bb99ad68bb44 None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:38.393459 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecebcf-d977-44fc-a151-bb99ad68bb44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:38.395595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ecebcf-d977-44fc-a151-bb99ad68bb44 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:38.416990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 177/360] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:38.416990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:38.418953 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecebcf-d977-44fc-a151-bb99ad68bb44 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:38.432815 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffccb7-8324-40a6-90e7-6e803400cd2c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:38.473601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bfd42-e913-462a-8764-2f8cd4b2a371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:38.474711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bfd42-e913-462a-8764-2f8cd4b2a371 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:38.475301 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bfd42-e913-462a-8764-2f8cd4b2a371 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:38.475769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bfd42-e913-462a-8764-2f8cd4b2a371 None admin] PATH_INFO: `/v3/groups/362a436695ce418b9853d511fd6b8243` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:38.477178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bfd42-e913-462a-8764-2f8cd4b2a371 None admin] RBAC: Authorizing `identity:get_group(group_id=362a436695ce418b9853d511fd6b8243)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:38.478999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bfd42-e913-462a-8764-2f8cd4b2a371 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:38.484718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 184/361] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:38 2022] GET /identity/v3/groups/362a436695ce418b9853d511fd6b8243 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:38.486453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2388483-0902-47ab-b039-789dbf435f78 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:38.486928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2388483-0902-47ab-b039-789dbf435f78 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:38.487471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2388483-0902-47ab-b039-789dbf435f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:38.509064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e2388483-0902-47ab-b039-789dbf435f78 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:38.511171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e2388483-0902-47ab-b039-789dbf435f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:38.529004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecebcf-d977-44fc-a151-bb99ad68bb44 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:38.531055 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:38.569313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aea577-9ec3-4dec-a581-40452ceb66ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:38.569920 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aea577-9ec3-4dec-a581-40452ceb66ed None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:38.570194 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aea577-9ec3-4dec-a581-40452ceb66ed None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:38.570460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aea577-9ec3-4dec-a581-40452ceb66ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:38.571185 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aea577-9ec3-4dec-a581-40452ceb66ed None admin] RBAC: Adding query filter params (role.id=712114868a39430b80ec9c19cb91c947, user.id=ba3276b8775b4117b3b8b13d06f76240, scope.system=all) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:38.571487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aea577-9ec3-4dec-a581-40452ceb66ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:38.573279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aea577-9ec3-4dec-a581-40452ceb66ed None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:38.586682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 178/362] 10.222.0.75 () {56 vars in 1308 bytes} [Fri Sep 23 15:04:38 2022] GET /identity/v3/role_assignments?user.id=ba3276b8775b4117b3b8b13d06f76240&scope.system=all&role.id=712114868a39430b80ec9c19cb91c947 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:38.587592 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aea577-9ec3-4dec-a581-40452ceb66ed None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:38.616315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 185/363] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:38.618219 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2388483-0902-47ab-b039-789dbf435f78 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:38.675145 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:38.739463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aea577-9ec3-4dec-a581-40452ceb66ed None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:38.763740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2388483-0902-47ab-b039-789dbf435f78 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:38.788828 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731cac65-1a23-49cd-bae9-6c89fc0e0909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:38.790082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cac65-1a23-49cd-bae9-6c89fc0e0909 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:38.790556 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cac65-1a23-49cd-bae9-6c89fc0e0909 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:38.791765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cac65-1a23-49cd-bae9-6c89fc0e0909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:38.806178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078575c5-040d-46d6-8a9c-3a0a14f21c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:38.808231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078575c5-040d-46d6-8a9c-3a0a14f21c53 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:38.808231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078575c5-040d-46d6-8a9c-3a0a14f21c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:38.808231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078575c5-040d-46d6-8a9c-3a0a14f21c53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:38.808504 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078575c5-040d-46d6-8a9c-3a0a14f21c53 None admin] RBAC: Adding query filter params (group.id=362a436695ce418b9853d511fd6b8243, role.id=704c67b455b54fb48089faef4aba8ccd, scope.project.id=9edb599ff8bb4c67b467b712ea2dc700) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:38.809873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078575c5-040d-46d6-8a9c-3a0a14f21c53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:38.811507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078575c5-040d-46d6-8a9c-3a0a14f21c53 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:38.823332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 186/364] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Sep 23 15:04:38 2022] GET /identity/v3/role_assignments?group.id=362a436695ce418b9853d511fd6b8243&scope.project.id=9edb599ff8bb4c67b467b712ea2dc700&role.id=704c67b455b54fb48089faef4aba8ccd => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:38.823332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078575c5-040d-46d6-8a9c-3a0a14f21c53 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:38.849903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:38.885581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731cac65-1a23-49cd-bae9-6c89fc0e0909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:38.887507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731cac65-1a23-49cd-bae9-6c89fc0e0909 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:38.997032 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:39.026840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078575c5-040d-46d6-8a9c-3a0a14f21c53 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:39.074159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff1259-877e-455f-a0ba-53b9449533f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:39.074159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff1259-877e-455f-a0ba-53b9449533f4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:39.074159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff1259-877e-455f-a0ba-53b9449533f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:39.074159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff1259-877e-455f-a0ba-53b9449533f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:39.077237 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 179/365] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:39.079761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731cac65-1a23-49cd-bae9-6c89fc0e0909 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:39.086523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff1259-877e-455f-a0ba-53b9449533f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-5eff1259-877e-455f-a0ba-53b9449533f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:39.093352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:39.094385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 187/366] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:04:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:39.096420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff1259-877e-455f-a0ba-53b9449533f4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:39.211244 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:39.240980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731cac65-1a23-49cd-bae9-6c89fc0e0909 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:39.253709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff1259-877e-455f-a0ba-53b9449533f4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:39.289651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ec40c-b8e2-4d2b-b78c-13fa92b6a0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:39.290699 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ec40c-b8e2-4d2b-b78c-13fa92b6a0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:39.291257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ec40c-b8e2-4d2b-b78c-13fa92b6a0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:39.291665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ec40c-b8e2-4d2b-b78c-13fa92b6a0d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:39.301659 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25b587-b149-4a5e-a009-a7fb52419dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:39.302506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25b587-b149-4a5e-a009-a7fb52419dcf None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:39.303176 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25b587-b149-4a5e-a009-a7fb52419dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:39.303453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25b587-b149-4a5e-a009-a7fb52419dcf None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:39.305772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25b587-b149-4a5e-a009-a7fb52419dcf None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:39.306366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ec40c-b8e2-4d2b-b78c-13fa92b6a0d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:39.307854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25b587-b149-4a5e-a009-a7fb52419dcf None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:39.309633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 188/367] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:39 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:39.313883 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25b587-b149-4a5e-a009-a7fb52419dcf None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-860ec40c-b8e2-4d2b-b78c-13fa92b6a0d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:39.318675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:39.319564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:39.320332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:39.320332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:39.320332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:39.320332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:39.320332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:39.320332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:39.320332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:39.320332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:39.320332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:39.320332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:39.320332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:39.320332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:39.323705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 180/368] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:04:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:39.326533 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ec40c-b8e2-4d2b-b78c-13fa92b6a0d1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:39.349773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:39.428320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25b587-b149-4a5e-a009-a7fb52419dcf None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:39.451861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ec40c-b8e2-4d2b-b78c-13fa92b6a0d1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:39.464658 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87d233-29c5-4cf6-ad7d-75ba8104c14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:39.465680 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87d233-29c5-4cf6-ad7d-75ba8104c14d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:39.466166 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87d233-29c5-4cf6-ad7d-75ba8104c14d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:39.466625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87d233-29c5-4cf6-ad7d-75ba8104c14d None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:39.467572 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87d233-29c5-4cf6-ad7d-75ba8104c14d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:39.468066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87d233-29c5-4cf6-ad7d-75ba8104c14d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:39.469658 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87d233-29c5-4cf6-ad7d-75ba8104c14d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:39.478485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 189/369] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:04:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:39.480287 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87d233-29c5-4cf6-ad7d-75ba8104c14d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:39.490012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:39.508555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f1a2f-badd-4ab4-b04a-9b99ddf88405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:39.509234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f1a2f-badd-4ab4-b04a-9b99ddf88405 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:39.509577 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f1a2f-badd-4ab4-b04a-9b99ddf88405 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:39.509875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f1a2f-badd-4ab4-b04a-9b99ddf88405 None admin] PATH_INFO: `/v3/groups/362a436695ce418b9853d511fd6b8243` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:39.512320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f1a2f-badd-4ab4-b04a-9b99ddf88405 None admin] RBAC: Authorizing `identity:get_group(group_id=362a436695ce418b9853d511fd6b8243)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:39.514094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f1a2f-badd-4ab4-b04a-9b99ddf88405 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:39.516034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 181/370] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:39 2022] GET /identity/v3/groups/362a436695ce418b9853d511fd6b8243 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:39.516850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f1a2f-badd-4ab4-b04a-9b99ddf88405 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:39.607330 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87d233-29c5-4cf6-ad7d-75ba8104c14d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:39.618571 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:39.639692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f1a2f-badd-4ab4-b04a-9b99ddf88405 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:39.648256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260de71-9c9f-4d5a-b006-789b950bc087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:39.649139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260de71-9c9f-4d5a-b006-789b950bc087 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:39.649571 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260de71-9c9f-4d5a-b006-789b950bc087 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:39.650157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260de71-9c9f-4d5a-b006-789b950bc087 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:39.651055 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260de71-9c9f-4d5a-b006-789b950bc087 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:39.651555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260de71-9c9f-4d5a-b006-789b950bc087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:39.653215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260de71-9c9f-4d5a-b006-789b950bc087 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:39.662629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 190/371] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:39.663705 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260de71-9c9f-4d5a-b006-789b950bc087 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:39.679342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed7878-1b0c-4262-8f74-eee87dab00d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:39.680064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed7878-1b0c-4262-8f74-eee87dab00d2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:39.680331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed7878-1b0c-4262-8f74-eee87dab00d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:39.680619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed7878-1b0c-4262-8f74-eee87dab00d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:39.739943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:39.764236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed7878-1b0c-4262-8f74-eee87dab00d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:39.764437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed7878-1b0c-4262-8f74-eee87dab00d2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:39.783522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260de71-9c9f-4d5a-b006-789b950bc087 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:39.819815 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc4006-2c76-41d2-ab7a-6f3e295e01de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:39.820703 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc4006-2c76-41d2-ab7a-6f3e295e01de None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:39.821222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc4006-2c76-41d2-ab7a-6f3e295e01de None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:39.821686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc4006-2c76-41d2-ab7a-6f3e295e01de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:39.822646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc4006-2c76-41d2-ab7a-6f3e295e01de None admin] RBAC: Adding query filter params (group.id=362a436695ce418b9853d511fd6b8243, role.id=2921a7146fd9408396ca98a4be4ecd3f, scope.project.id=117d6fb504534ef2bd643528f241dffa) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:39.823161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc4006-2c76-41d2-ab7a-6f3e295e01de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:39.824842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cc4006-2c76-41d2-ab7a-6f3e295e01de None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:39.832163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 191/372] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Sep 23 15:04:39 2022] GET /identity/v3/role_assignments?group.id=362a436695ce418b9853d511fd6b8243&scope.project.id=117d6fb504534ef2bd643528f241dffa&role.id=2921a7146fd9408396ca98a4be4ecd3f => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:39.833321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc4006-2c76-41d2-ab7a-6f3e295e01de None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:39.858571 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:39.886274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 182/373] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:39.889335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed7878-1b0c-4262-8f74-eee87dab00d2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:40.013076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed7878-1b0c-4262-8f74-eee87dab00d2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:40.044931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc4006-2c76-41d2-ab7a-6f3e295e01de None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:40.051657 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590af7e4-fd2b-4198-a019-d910c26c7a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:40.052304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590af7e4-fd2b-4198-a019-d910c26c7a01 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:40.059476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590af7e4-fd2b-4198-a019-d910c26c7a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:40.059476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590af7e4-fd2b-4198-a019-d910c26c7a01 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:40.059476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590af7e4-fd2b-4198-a019-d910c26c7a01 None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:40.059476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590af7e4-fd2b-4198-a019-d910c26c7a01 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:40.059476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 183/374] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:39 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:40.064387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590af7e4-fd2b-4198-a019-d910c26c7a01 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:40.114088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:40.125172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:40.126431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85441474-de5e-4e63-b108-f0972235052d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:40.127321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85441474-de5e-4e63-b108-f0972235052d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:40.127799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85441474-de5e-4e63-b108-f0972235052d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:40.128261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85441474-de5e-4e63-b108-f0972235052d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:40.183553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590af7e4-fd2b-4198-a019-d910c26c7a01 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:40.224088 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9518e-e1fe-4649-ad8f-da94c4935748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:40.224530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9518e-e1fe-4649-ad8f-da94c4935748 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:40.224832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9518e-e1fe-4649-ad8f-da94c4935748 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:40.225164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db9518e-e1fe-4649-ad8f-da94c4935748 None admin] PATH_INFO: `/v3/groups/362a436695ce418b9853d511fd6b8243` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:40.226352 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9518e-e1fe-4649-ad8f-da94c4935748 None admin] RBAC: Authorizing `identity:get_group(group_id=362a436695ce418b9853d511fd6b8243)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:40.228103 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db9518e-e1fe-4649-ad8f-da94c4935748 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:40.234598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 184/375] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:40 2022] GET /identity/v3/groups/362a436695ce418b9853d511fd6b8243 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:40.237613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9518e-e1fe-4649-ad8f-da94c4935748 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:40.255817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85441474-de5e-4e63-b108-f0972235052d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:40.257693 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85441474-de5e-4e63-b108-f0972235052d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:40.355411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db9518e-e1fe-4649-ad8f-da94c4935748 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:40.375326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:40.395012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cae074-2b4f-4601-8907-2fdbcacfcd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:40.395619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cae074-2b4f-4601-8907-2fdbcacfcd15 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:40.395890 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cae074-2b4f-4601-8907-2fdbcacfcd15 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:40.396524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cae074-2b4f-4601-8907-2fdbcacfcd15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:40.396973 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cae074-2b4f-4601-8907-2fdbcacfcd15 None admin] RBAC: Adding query filter params (group.id=362a436695ce418b9853d511fd6b8243, role.id=704c67b455b54fb48089faef4aba8ccd, scope.project.id=117d6fb504534ef2bd643528f241dffa) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:40.397277 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cae074-2b4f-4601-8907-2fdbcacfcd15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:40.398713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cae074-2b4f-4601-8907-2fdbcacfcd15 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:40.408737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 185/376] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Sep 23 15:04:40 2022] GET /identity/v3/role_assignments?group.id=362a436695ce418b9853d511fd6b8243&scope.project.id=117d6fb504534ef2bd643528f241dffa&role.id=704c67b455b54fb48089faef4aba8ccd => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:40.423641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 192/377] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:40.429292 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cae074-2b4f-4601-8907-2fdbcacfcd15 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:40.572902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cae074-2b4f-4601-8907-2fdbcacfcd15 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:40.611054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567658-4f08-41cb-a0e1-27fd9ab6ab98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:40.611697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567658-4f08-41cb-a0e1-27fd9ab6ab98 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:40.611969 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567658-4f08-41cb-a0e1-27fd9ab6ab98 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:40.612236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567658-4f08-41cb-a0e1-27fd9ab6ab98 None admin] PATH_INFO: `/v3/projects/a6e3ae74e78241ac8bfc0ae2c7103bec` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:40.613404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26567658-4f08-41cb-a0e1-27fd9ab6ab98 None admin] RBAC: Authorizing `identity:get_project(project_id=a6e3ae74e78241ac8bfc0ae2c7103bec)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:40.614822 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26567658-4f08-41cb-a0e1-27fd9ab6ab98 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:40.616858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 186/378] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:40 2022] GET /identity/v3/projects/a6e3ae74e78241ac8bfc0ae2c7103bec => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:40.622572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567658-4f08-41cb-a0e1-27fd9ab6ab98 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:40.623161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08c978-e97d-4d03-bab8-e02e885f3782 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:40.623507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08c978-e97d-4d03-bab8-e02e885f3782 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:40.623624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08c978-e97d-4d03-bab8-e02e885f3782 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:40.625228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 193/379] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:40.631371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49f50a-7b57-4b17-a751-ac1d172683d0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:40.631641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49f50a-7b57-4b17-a751-ac1d172683d0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:40.631834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49f50a-7b57-4b17-a751-ac1d172683d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:40.654971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7b49f50a-7b57-4b17-a751-ac1d172683d0 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:40.656461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7b49f50a-7b57-4b17-a751-ac1d172683d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:40.698382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:40.710259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:40.754070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 194/380] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:40.783323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567658-4f08-41cb-a0e1-27fd9ab6ab98 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:40.819848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44bfd9-6af7-4911-a02b-724b4272a60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:40.820474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44bfd9-6af7-4911-a02b-724b4272a60f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:40.820774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44bfd9-6af7-4911-a02b-724b4272a60f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:40.821040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44bfd9-6af7-4911-a02b-724b4272a60f None admin] PATH_INFO: `/v3/groups/3a0dd69a67f24a65ad58847e94c8c867` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:40.826704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44bfd9-6af7-4911-a02b-724b4272a60f None admin] RBAC: Authorizing `identity:get_group(group_id=3a0dd69a67f24a65ad58847e94c8c867)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:40.828133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44bfd9-6af7-4911-a02b-724b4272a60f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:40.830144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 187/381] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:40 2022] GET /identity/v3/groups/3a0dd69a67f24a65ad58847e94c8c867 => generated 229 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:40.836096 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49f50a-7b57-4b17-a751-ac1d172683d0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:40.952882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49f50a-7b57-4b17-a751-ac1d172683d0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:40.993186 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1e338-8ef4-4c58-b54e-352853324ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:40.993915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1e338-8ef4-4c58-b54e-352853324ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:40.994325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1e338-8ef4-4c58-b54e-352853324ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:40.994859 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1e338-8ef4-4c58-b54e-352853324ed6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:40.995635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1e338-8ef4-4c58-b54e-352853324ed6 None admin] RBAC: Adding query filter params (group.id=3a0dd69a67f24a65ad58847e94c8c867, role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=a6e3ae74e78241ac8bfc0ae2c7103bec) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:40.995939 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1e338-8ef4-4c58-b54e-352853324ed6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:40.997417 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1e338-8ef4-4c58-b54e-352853324ed6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:41.007339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 195/382] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Sep 23 15:04:40 2022] GET /identity/v3/role_assignments?group.id=3a0dd69a67f24a65ad58847e94c8c867&scope.project.id=a6e3ae74e78241ac8bfc0ae2c7103bec&role.id=ae444a489623434999d758b71ae2dc87 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:41.129344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44bfd9-6af7-4911-a02b-724b4272a60f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:41.256243 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44bfd9-6af7-4911-a02b-724b4272a60f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:41.274287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:41.298978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f205c-5066-4ce0-b74a-b08ef94afa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:41.299658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f205c-5066-4ce0-b74a-b08ef94afa0b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:41.299937 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f205c-5066-4ce0-b74a-b08ef94afa0b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:41.300199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f205c-5066-4ce0-b74a-b08ef94afa0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:41.312174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f205c-5066-4ce0-b74a-b08ef94afa0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-1e4f205c-5066-4ce0-b74a-b08ef94afa0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:41.317612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:41.319713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:41.321701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:41.321701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:41.321701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:41.321701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:41.321701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:41.321701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:41.321701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:41.321701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:41.321701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:41.321701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:41.321701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:41.321701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:41.321701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 188/383] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:41.328881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1e338-8ef4-4c58-b54e-352853324ed6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:41.386258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:41.484891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1e338-8ef4-4c58-b54e-352853324ed6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:41.532235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfe9ae-0c16-4ecf-9479-e9241586d18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:41.532834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfe9ae-0c16-4ecf-9479-e9241586d18b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:41.533115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfe9ae-0c16-4ecf-9479-e9241586d18b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:41.533383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfe9ae-0c16-4ecf-9479-e9241586d18b None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:41.534056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfe9ae-0c16-4ecf-9479-e9241586d18b None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:41.534354 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfe9ae-0c16-4ecf-9479-e9241586d18b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:41.535730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dfe9ae-0c16-4ecf-9479-e9241586d18b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:41.549485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 196/384] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:41.554714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f205c-5066-4ce0-b74a-b08ef94afa0b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:41.695588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f205c-5066-4ce0-b74a-b08ef94afa0b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:41.734910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f51bf7-215b-4a90-ab40-0761946d79a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:41.735610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f51bf7-215b-4a90-ab40-0761946d79a6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:41.735886 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f51bf7-215b-4a90-ab40-0761946d79a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:41.736155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f51bf7-215b-4a90-ab40-0761946d79a6 None admin] PATH_INFO: `/v3/groups/362a436695ce418b9853d511fd6b8243` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:41.741245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f51bf7-215b-4a90-ab40-0761946d79a6 None admin] RBAC: Authorizing `identity:get_group(group_id=362a436695ce418b9853d511fd6b8243)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:41.742673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f51bf7-215b-4a90-ab40-0761946d79a6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:41.744677 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 189/385] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:41 2022] GET /identity/v3/groups/362a436695ce418b9853d511fd6b8243 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:41.750396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfe9ae-0c16-4ecf-9479-e9241586d18b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:41.861891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:41.883433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfe9ae-0c16-4ecf-9479-e9241586d18b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:41.925333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd742e-757b-4746-a32f-8cb6ebc9c1df None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:41.925684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd742e-757b-4746-a32f-8cb6ebc9c1df None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:41.925862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd742e-757b-4746-a32f-8cb6ebc9c1df None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:41.927315 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 190/386] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:04:41.931076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e616-3dc6-4a3d-9274-a9cef4bb9e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:41.932010 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e616-3dc6-4a3d-9274-a9cef4bb9e05 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:41.932478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e616-3dc6-4a3d-9274-a9cef4bb9e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:41.932777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e616-3dc6-4a3d-9274-a9cef4bb9e05 None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:41.937482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271a27c-ff72-46a0-8b4f-f00dbe6e4673 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:41.937759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271a27c-ff72-46a0-8b4f-f00dbe6e4673 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:41.938024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271a27c-ff72-46a0-8b4f-f00dbe6e4673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:41.938781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2e616-3dc6-4a3d-9274-a9cef4bb9e05 None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:41.940227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2e616-3dc6-4a3d-9274-a9cef4bb9e05 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:41.942029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 197/387] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:41 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:41.949628 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e616-3dc6-4a3d-9274-a9cef4bb9e05 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:41.972421 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9271a27c-ff72-46a0-8b4f-f00dbe6e4673 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:41.973638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9271a27c-ff72-46a0-8b4f-f00dbe6e4673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:42.044403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:42.093775 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 191/388] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:42.132415 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e616-3dc6-4a3d-9274-a9cef4bb9e05 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:42.187272 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8abde7-19bd-4cfe-a1c3-8fc28ab5674e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:42.187956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8abde7-19bd-4cfe-a1c3-8fc28ab5674e None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:42.188230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8abde7-19bd-4cfe-a1c3-8fc28ab5674e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:42.188495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8abde7-19bd-4cfe-a1c3-8fc28ab5674e None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700/groups/362a436695ce418b9853d511fd6b8243/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:42.190574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8abde7-19bd-4cfe-a1c3-8fc28ab5674e None admin] RBAC: Authorizing `identity:create_grant(project_id=9edb599ff8bb4c67b467b712ea2dc700, group_id=362a436695ce418b9853d511fd6b8243, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:42.192034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8abde7-19bd-4cfe-a1c3-8fc28ab5674e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:42.210963 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 198/389] 10.222.0.75 () {58 vars in 1442 bytes} [Fri Sep 23 15:04:41 2022] PUT /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700/groups/362a436695ce418b9853d511fd6b8243/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:42.516501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271a27c-ff72-46a0-8b4f-f00dbe6e4673 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:42.632599 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271a27c-ff72-46a0-8b4f-f00dbe6e4673 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:42.671701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33032070-d0ed-4fb6-99c0-159efb2992b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:42.672356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032070-d0ed-4fb6-99c0-159efb2992b1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:42.672645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032070-d0ed-4fb6-99c0-159efb2992b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:42.672927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33032070-d0ed-4fb6-99c0-159efb2992b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:42.687993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33032070-d0ed-4fb6-99c0-159efb2992b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-33032070-d0ed-4fb6-99c0-159efb2992b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:42.693102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:42.695040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:42.696647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:42.696647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:42.696647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:42.696647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:42.696647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:42.696647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:42.696647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:42.696647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:42.696647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:42.696647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:42.696647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:42.696647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:42.696647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 192/390] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:04:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:42.700756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8abde7-19bd-4cfe-a1c3-8fc28ab5674e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:42.721531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:42.816534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d335d-8929-4d70-bdca-0ff390647f9c None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:42.816897 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d335d-8929-4d70-bdca-0ff390647f9c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:42.817146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d335d-8929-4d70-bdca-0ff390647f9c None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:42.818452 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 193/391] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:42.823174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8abde7-19bd-4cfe-a1c3-8fc28ab5674e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:42.824190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5f3a7-14ee-48d9-ac30-05074399fd07 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:42.824472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5f3a7-14ee-48d9-ac30-05074399fd07 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:42.824787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5f3a7-14ee-48d9-ac30-05074399fd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:42.845067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f3f5f3a7-14ee-48d9-ac30-05074399fd07 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:42.846257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f3f5f3a7-14ee-48d9-ac30-05074399fd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:42.872556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34bae2-95f2-4004-84ed-cd73af34f547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:42.873201 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34bae2-95f2-4004-84ed-cd73af34f547 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:42.873739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34bae2-95f2-4004-84ed-cd73af34f547 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:42.874018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a34bae2-95f2-4004-84ed-cd73af34f547 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:42.875101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34bae2-95f2-4004-84ed-cd73af34f547 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:42.875401 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34bae2-95f2-4004-84ed-cd73af34f547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:42.876888 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a34bae2-95f2-4004-84ed-cd73af34f547 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:42.892511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:42.894032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 199/392] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:04:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:42.901345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34bae2-95f2-4004-84ed-cd73af34f547 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:42.981431 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 194/393] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:43.019858 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:43.042749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a34bae2-95f2-4004-84ed-cd73af34f547 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:43.081533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89cdf6-c4f0-4f17-99d5-d4e27c9d5670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:43.082089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89cdf6-c4f0-4f17-99d5-d4e27c9d5670 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:43.082364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89cdf6-c4f0-4f17-99d5-d4e27c9d5670 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:43.082635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89cdf6-c4f0-4f17-99d5-d4e27c9d5670 None admin] PATH_INFO: `/v3/groups/362a436695ce418b9853d511fd6b8243` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:43.083746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89cdf6-c4f0-4f17-99d5-d4e27c9d5670 None admin] RBAC: Authorizing `identity:get_group(group_id=362a436695ce418b9853d511fd6b8243)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:43.085239 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89cdf6-c4f0-4f17-99d5-d4e27c9d5670 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:43.087001 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 200/394] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:42 2022] GET /identity/v3/groups/362a436695ce418b9853d511fd6b8243 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:43.092495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5f3a7-14ee-48d9-ac30-05074399fd07 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:43.207151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5f3a7-14ee-48d9-ac30-05074399fd07 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:43.245285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a468869-24fb-484b-b700-2bfd73edcca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:43.245984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a468869-24fb-484b-b700-2bfd73edcca9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:43.246301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a468869-24fb-484b-b700-2bfd73edcca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:43.246574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a468869-24fb-484b-b700-2bfd73edcca9 None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:43.247841 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a468869-24fb-484b-b700-2bfd73edcca9 None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:43.249397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a468869-24fb-484b-b700-2bfd73edcca9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:43.251223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 195/395] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:43 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:43.256873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89cdf6-c4f0-4f17-99d5-d4e27c9d5670 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:43.339697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a468869-24fb-484b-b700-2bfd73edcca9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:43.375128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:43.394537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89cdf6-c4f0-4f17-99d5-d4e27c9d5670 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:43.461310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a468869-24fb-484b-b700-2bfd73edcca9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:43.483517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0717052-a755-45b7-b763-c9638fe41c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:43.484548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0717052-a755-45b7-b763-c9638fe41c7f None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:43.485060 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0717052-a755-45b7-b763-c9638fe41c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:43.485518 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0717052-a755-45b7-b763-c9638fe41c7f None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700/groups/362a436695ce418b9853d511fd6b8243/roles/704c67b455b54fb48089faef4aba8ccd` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:43.490369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0717052-a755-45b7-b763-c9638fe41c7f None admin] RBAC: Authorizing `identity:create_grant(project_id=9edb599ff8bb4c67b467b712ea2dc700, group_id=362a436695ce418b9853d511fd6b8243, role_id=704c67b455b54fb48089faef4aba8ccd)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:43.492064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0717052-a755-45b7-b763-c9638fe41c7f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:43.519207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 201/396] 10.222.0.75 () {58 vars in 1442 bytes} [Fri Sep 23 15:04:43 2022] PUT /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700/groups/362a436695ce418b9853d511fd6b8243/roles/704c67b455b54fb48089faef4aba8ccd => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:43.541220 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa421cab-8e90-482c-9f1f-e45747489328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:43.542369 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa421cab-8e90-482c-9f1f-e45747489328 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:43.542841 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa421cab-8e90-482c-9f1f-e45747489328 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:43.543305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa421cab-8e90-482c-9f1f-e45747489328 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:43.563321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa421cab-8e90-482c-9f1f-e45747489328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-fa421cab-8e90-482c-9f1f-e45747489328 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:43.574059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:43.575025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:43.576297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:43.576297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:43.576297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:43.576297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:43.576297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:43.576297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:43.576297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:43.576297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:43.576297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:43.576297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:43.576297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:43.576297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:43.576297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 196/397] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:43.583049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0717052-a755-45b7-b763-c9638fe41c7f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:43.594493 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:43.648549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189ea5f-19e7-41d3-89ea-15f296dd27c3 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:43.650062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189ea5f-19e7-41d3-89ea-15f296dd27c3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:43.650827 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189ea5f-19e7-41d3-89ea-15f296dd27c3 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:43.652352 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 197/398] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:43.659241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe39648-d4b0-40c8-b903-b352ca2c0841 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:43.659527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe39648-d4b0-40c8-b903-b352ca2c0841 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:43.659768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe39648-d4b0-40c8-b903-b352ca2c0841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:43.691801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-abe39648-d4b0-40c8-b903-b352ca2c0841 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:43.694234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-abe39648-d4b0-40c8-b903-b352ca2c0841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:43.778253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0717052-a755-45b7-b763-c9638fe41c7f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:43.806764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 198/399] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:43.824558 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c578b-4f8b-4345-9401-5f66c3767fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:43.825215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c578b-4f8b-4345-9401-5f66c3767fdd None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:43.825516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c578b-4f8b-4345-9401-5f66c3767fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:43.825783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c578b-4f8b-4345-9401-5f66c3767fdd None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:43.826454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c578b-4f8b-4345-9401-5f66c3767fdd None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:43.826751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c578b-4f8b-4345-9401-5f66c3767fdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:43.828113 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c578b-4f8b-4345-9401-5f66c3767fdd None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:43.837037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 202/400] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:43.843521 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe39648-d4b0-40c8-b903-b352ca2c0841 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:43.868908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:43.952456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe39648-d4b0-40c8-b903-b352ca2c0841 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:43.980165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288adcaa-c863-4fc4-b392-518ac50f009e None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:43.980470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288adcaa-c863-4fc4-b392-518ac50f009e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:43.980720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288adcaa-c863-4fc4-b392-518ac50f009e None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:43.982265 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 203/401] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:04:43.988076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e734e-cb1d-4b7d-9c9c-d01c0733570b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:43.988650 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e734e-cb1d-4b7d-9c9c-d01c0733570b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:43.989131 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e734e-cb1d-4b7d-9c9c-d01c0733570b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:44.003390 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0df7fc-d93f-4443-82b3-c44fc48d8d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:44.004030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0df7fc-d93f-4443-82b3-c44fc48d8d78 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:44.004297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0df7fc-d93f-4443-82b3-c44fc48d8d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:44.004558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0df7fc-d93f-4443-82b3-c44fc48d8d78 None admin] PATH_INFO: `/v3/groups/362a436695ce418b9853d511fd6b8243` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:44.005697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0df7fc-d93f-4443-82b3-c44fc48d8d78 None admin] RBAC: Authorizing `identity:get_group(group_id=362a436695ce418b9853d511fd6b8243)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:44.007116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0df7fc-d93f-4443-82b3-c44fc48d8d78 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:44.008722 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 199/402] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:43 2022] GET /identity/v3/groups/362a436695ce418b9853d511fd6b8243 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:44.010226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f77e734e-cb1d-4b7d-9c9c-d01c0733570b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:44.011651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f77e734e-cb1d-4b7d-9c9c-d01c0733570b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:44.015987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0df7fc-d93f-4443-82b3-c44fc48d8d78 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:44.082996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:44.089998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 204/403] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:44.122215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:44.134875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0df7fc-d93f-4443-82b3-c44fc48d8d78 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:44.170512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaa94f-833c-4bfb-adc3-51dbdb05bf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:44.171146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaa94f-833c-4bfb-adc3-51dbdb05bf63 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:44.171413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaa94f-833c-4bfb-adc3-51dbdb05bf63 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:44.171679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaa94f-833c-4bfb-adc3-51dbdb05bf63 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:44.172823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaa94f-833c-4bfb-adc3-51dbdb05bf63 None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:44.174309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaa94f-833c-4bfb-adc3-51dbdb05bf63 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:44.176237 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 200/404] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:44 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:44.182469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e734e-cb1d-4b7d-9c9c-d01c0733570b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:44.221866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaa94f-833c-4bfb-adc3-51dbdb05bf63 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:44.353907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e734e-cb1d-4b7d-9c9c-d01c0733570b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:44.387082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaa94f-833c-4bfb-adc3-51dbdb05bf63 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:44.397820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7312378-6c7f-4eab-907d-1fb5e24969af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:44.398487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7312378-6c7f-4eab-907d-1fb5e24969af None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:44.398757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7312378-6c7f-4eab-907d-1fb5e24969af None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:44.399020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7312378-6c7f-4eab-907d-1fb5e24969af None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa/groups/362a436695ce418b9853d511fd6b8243/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:44.401384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7312378-6c7f-4eab-907d-1fb5e24969af None admin] RBAC: Authorizing `identity:create_grant(project_id=117d6fb504534ef2bd643528f241dffa, group_id=362a436695ce418b9853d511fd6b8243, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:44.402920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7312378-6c7f-4eab-907d-1fb5e24969af None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:44.418904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 205/405] 10.222.0.75 () {58 vars in 1442 bytes} [Fri Sep 23 15:04:44 2022] PUT /identity/v3/projects/117d6fb504534ef2bd643528f241dffa/groups/362a436695ce418b9853d511fd6b8243/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:44.419968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7312378-6c7f-4eab-907d-1fb5e24969af None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:44.427096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f832b-c584-4439-934e-3c3b6daa0e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:44.428263 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f832b-c584-4439-934e-3c3b6daa0e32 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:44.428795 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f832b-c584-4439-934e-3c3b6daa0e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:44.429975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f832b-c584-4439-934e-3c3b6daa0e32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:44.446065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f832b-c584-4439-934e-3c3b6daa0e32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-ab0f832b-c584-4439-934e-3c3b6daa0e32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:44.452663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:44.453704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:44.454500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:44.454500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:44.454500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:44.454500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:44.454500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:44.454500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:44.454500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:44.454500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:44.454500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:44.454500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:44.454500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:44.454500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:44.456970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 201/406] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:04:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:44.465043 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f832b-c584-4439-934e-3c3b6daa0e32 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:44.491342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:44.499181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:44.594625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7312378-6c7f-4eab-907d-1fb5e24969af None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:44.624736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f832b-c584-4439-934e-3c3b6daa0e32 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:44.663868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086c8e7-058b-4a66-bb2b-7f748f0601ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:44.664834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086c8e7-058b-4a66-bb2b-7f748f0601ad None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:44.665732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086c8e7-058b-4a66-bb2b-7f748f0601ad None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:44.666520 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086c8e7-058b-4a66-bb2b-7f748f0601ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:44.678427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086c8e7-058b-4a66-bb2b-7f748f0601ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-f086c8e7-058b-4a66-bb2b-7f748f0601ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:44.682875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:44.683965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:44.684934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:44.684934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:44.684934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:44.684934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:44.684934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:44.684934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:44.684934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:44.684934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:44.684934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:44.684934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:44.684934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:44.684934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:44.685739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 206/407] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:04:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:44.691086 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086c8e7-058b-4a66-bb2b-7f748f0601ad None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:44.693478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723e7c4-928b-431b-8b43-9795a94ba9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:44.694288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723e7c4-928b-431b-8b43-9795a94ba9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:44.694704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723e7c4-928b-431b-8b43-9795a94ba9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:44.695159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723e7c4-928b-431b-8b43-9795a94ba9d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:44.697983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723e7c4-928b-431b-8b43-9795a94ba9d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:44.698525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723e7c4-928b-431b-8b43-9795a94ba9d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:44.700125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723e7c4-928b-431b-8b43-9795a94ba9d1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:44.713359 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 202/408] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:04:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:44.721719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723e7c4-928b-431b-8b43-9795a94ba9d1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:44.853798 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086c8e7-058b-4a66-bb2b-7f748f0601ad None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:44.867857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:44.891091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:44.914916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcea09d-ce38-4720-baec-d1a99076f96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:44.917639 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcea09d-ce38-4720-baec-d1a99076f96b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:44.918052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcea09d-ce38-4720-baec-d1a99076f96b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:44.918460 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcea09d-ce38-4720-baec-d1a99076f96b None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:44.919212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcea09d-ce38-4720-baec-d1a99076f96b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:44.919512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcea09d-ce38-4720-baec-d1a99076f96b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:44.920186 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723e7c4-928b-431b-8b43-9795a94ba9d1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:44.920954 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcea09d-ce38-4720-baec-d1a99076f96b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:44.944211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 207/409] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:44.949481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcea09d-ce38-4720-baec-d1a99076f96b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:44.964011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9ea1c-cfb9-48c0-a102-0287689309cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:44.967690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9ea1c-cfb9-48c0-a102-0287689309cb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:44.968213 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9ea1c-cfb9-48c0-a102-0287689309cb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:44.968721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9ea1c-cfb9-48c0-a102-0287689309cb None admin] PATH_INFO: `/v3/groups/362a436695ce418b9853d511fd6b8243` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:44.971328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9ea1c-cfb9-48c0-a102-0287689309cb None admin] RBAC: Authorizing `identity:get_group(group_id=362a436695ce418b9853d511fd6b8243)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:44.974223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9ea1c-cfb9-48c0-a102-0287689309cb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:44.976191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 203/410] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:44 2022] GET /identity/v3/groups/362a436695ce418b9853d511fd6b8243 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:44.981795 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9ea1c-cfb9-48c0-a102-0287689309cb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:45.087024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcea09d-ce38-4720-baec-d1a99076f96b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:45.136341 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87445a43-d04d-491a-adea-19afc37fa1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:45.138427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87445a43-d04d-491a-adea-19afc37fa1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:45.139782 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87445a43-d04d-491a-adea-19afc37fa1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:45.140280 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87445a43-d04d-491a-adea-19afc37fa1f6 None admin] PATH_INFO: `/v3/groups/3a0dd69a67f24a65ad58847e94c8c867` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:45.142957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87445a43-d04d-491a-adea-19afc37fa1f6 None admin] RBAC: Authorizing `identity:get_group(group_id=3a0dd69a67f24a65ad58847e94c8c867)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:45.146172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87445a43-d04d-491a-adea-19afc37fa1f6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:45.149199 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 208/411] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:44 2022] GET /identity/v3/groups/3a0dd69a67f24a65ad58847e94c8c867 => generated 229 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:45.154862 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87445a43-d04d-491a-adea-19afc37fa1f6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:45.200618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9ea1c-cfb9-48c0-a102-0287689309cb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:45.249400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6479b-6783-4c82-8b25-ecea18663dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:45.251230 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6479b-6783-4c82-8b25-ecea18663dff None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:45.251500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6479b-6783-4c82-8b25-ecea18663dff None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:45.251985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6479b-6783-4c82-8b25-ecea18663dff None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:45.253955 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6479b-6783-4c82-8b25-ecea18663dff None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:45.257559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6479b-6783-4c82-8b25-ecea18663dff None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:45.259598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 204/412] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:44 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:45.269829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6479b-6783-4c82-8b25-ecea18663dff None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:45.283795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:45.294830 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:45.388367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6479b-6783-4c82-8b25-ecea18663dff None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:45.428087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdcba0-0d60-4c3f-8f44-ef5a0bdf8f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:45.429216 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdcba0-0d60-4c3f-8f44-ef5a0bdf8f96 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:45.429747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdcba0-0d60-4c3f-8f44-ef5a0bdf8f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:45.430019 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdcba0-0d60-4c3f-8f44-ef5a0bdf8f96 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa/groups/362a436695ce418b9853d511fd6b8243/roles/704c67b455b54fb48089faef4aba8ccd` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:45.433294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdcba0-0d60-4c3f-8f44-ef5a0bdf8f96 None admin] RBAC: Authorizing `identity:create_grant(project_id=117d6fb504534ef2bd643528f241dffa, group_id=362a436695ce418b9853d511fd6b8243, role_id=704c67b455b54fb48089faef4aba8ccd)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:45.434789 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdcba0-0d60-4c3f-8f44-ef5a0bdf8f96 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:45.434899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87445a43-d04d-491a-adea-19afc37fa1f6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:45.452999 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 205/413] 10.222.0.75 () {58 vars in 1442 bytes} [Fri Sep 23 15:04:45 2022] PUT /identity/v3/projects/117d6fb504534ef2bd643528f241dffa/groups/362a436695ce418b9853d511fd6b8243/roles/704c67b455b54fb48089faef4aba8ccd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:45.454171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c63428-15b8-4f91-af08-b03da70c7543 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:45.454446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c63428-15b8-4f91-af08-b03da70c7543 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:45.454711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c63428-15b8-4f91-af08-b03da70c7543 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:45.457062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 206/414] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:45.463324 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e28c56-145f-429b-866c-58e5303bd935 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:45.463598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e28c56-145f-429b-866c-58e5303bd935 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:45.463967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e28c56-145f-429b-866c-58e5303bd935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:45.489579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a6e28c56-145f-429b-866c-58e5303bd935 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:45.491141 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a6e28c56-145f-429b-866c-58e5303bd935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:45.510154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678bd92-78a1-400d-a654-0c35707684fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:45.510898 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678bd92-78a1-400d-a654-0c35707684fe None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:45.511178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678bd92-78a1-400d-a654-0c35707684fe None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:45.511599 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678bd92-78a1-400d-a654-0c35707684fe None admin] PATH_INFO: `/v3/projects/a6e3ae74e78241ac8bfc0ae2c7103bec` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:45.516003 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678bd92-78a1-400d-a654-0c35707684fe None admin] RBAC: Authorizing `identity:get_project(project_id=a6e3ae74e78241ac8bfc0ae2c7103bec)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:45.517607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4678bd92-78a1-400d-a654-0c35707684fe None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:45.519697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 209/415] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:45 2022] GET /identity/v3/projects/a6e3ae74e78241ac8bfc0ae2c7103bec => generated 344 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:45.526802 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678bd92-78a1-400d-a654-0c35707684fe None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:45.589270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 207/416] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:45.738780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678bd92-78a1-400d-a654-0c35707684fe None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:45.757751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:45.778190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa84bb7-eb3d-4868-bdcf-12134f20eae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:45.778952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa84bb7-eb3d-4868-bdcf-12134f20eae7 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:45.779295 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa84bb7-eb3d-4868-bdcf-12134f20eae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:45.779634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa84bb7-eb3d-4868-bdcf-12134f20eae7 None admin] PATH_INFO: `/v3/projects/a6e3ae74e78241ac8bfc0ae2c7103bec/groups/3a0dd69a67f24a65ad58847e94c8c867/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:45.781645 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa84bb7-eb3d-4868-bdcf-12134f20eae7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6e3ae74e78241ac8bfc0ae2c7103bec, group_id=3a0dd69a67f24a65ad58847e94c8c867, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:45.783152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa84bb7-eb3d-4868-bdcf-12134f20eae7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:45.800654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 210/417] 10.222.0.75 () {58 vars in 1442 bytes} [Fri Sep 23 15:04:45 2022] PUT /identity/v3/projects/a6e3ae74e78241ac8bfc0ae2c7103bec/groups/3a0dd69a67f24a65ad58847e94c8c867/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:04:45.927330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e28c56-145f-429b-866c-58e5303bd935 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:45.930739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:46.071983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e28c56-145f-429b-866c-58e5303bd935 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:46.113366 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9f4b1-8f7d-4ce7-ae95-af64fac37ade None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:46.113885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9f4b1-8f7d-4ce7-ae95-af64fac37ade None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:46.113885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9f4b1-8f7d-4ce7-ae95-af64fac37ade None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:46.115554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 211/418] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:46.120437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ff93d-5667-402f-9a0b-45e06123389c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:46.121118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276430a-4ba8-48ec-8494-71250b6cfec4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:46.121404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276430a-4ba8-48ec-8494-71250b6cfec4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:46.121680 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276430a-4ba8-48ec-8494-71250b6cfec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:46.122012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ff93d-5667-402f-9a0b-45e06123389c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:46.122505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ff93d-5667-402f-9a0b-45e06123389c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:46.122962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ff93d-5667-402f-9a0b-45e06123389c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:46.135395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ff93d-5667-402f-9a0b-45e06123389c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-bd8ff93d-5667-402f-9a0b-45e06123389c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:46.140569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:46.141482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:46.142301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:46.142301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:46.142301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:46.142301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:46.142301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:46.142301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:46.142301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:46.142301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:46.142301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:46.142301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:46.142301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:46.142301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:46.143601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a276430a-4ba8-48ec-8494-71250b6cfec4 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:46.144851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a276430a-4ba8-48ec-8494-71250b6cfec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:46.145335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 208/419] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:46.153596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ff93d-5667-402f-9a0b-45e06123389c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:46.219713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:46.256713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 212/420] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:46.268614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ff93d-5667-402f-9a0b-45e06123389c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:46.299749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dfe90-957e-4540-b32e-2ce79f4592e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:46.300759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dfe90-957e-4540-b32e-2ce79f4592e1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:46.301033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dfe90-957e-4540-b32e-2ce79f4592e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:46.301296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36dfe90-957e-4540-b32e-2ce79f4592e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:46.301959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36dfe90-957e-4540-b32e-2ce79f4592e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:46.302294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36dfe90-957e-4540-b32e-2ce79f4592e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:46.303655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36dfe90-957e-4540-b32e-2ce79f4592e1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:46.319066 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 209/421] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:46.326622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276430a-4ba8-48ec-8494-71250b6cfec4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:46.455220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276430a-4ba8-48ec-8494-71250b6cfec4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:46.462964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:46.493019 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d8ec3-37b2-4e38-b97e-8cd1e22ae1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:46.493690 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d8ec3-37b2-4e38-b97e-8cd1e22ae1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:46.494057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d8ec3-37b2-4e38-b97e-8cd1e22ae1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:46.494443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d8ec3-37b2-4e38-b97e-8cd1e22ae1a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:46.564434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d8ec3-37b2-4e38-b97e-8cd1e22ae1a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:46.566097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d8ec3-37b2-4e38-b97e-8cd1e22ae1a1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:46.678144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 213/422] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:46.685795 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dfe90-957e-4540-b32e-2ce79f4592e1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:46.730937 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d8ec3-37b2-4e38-b97e-8cd1e22ae1a1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:46.740415 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:46.859706 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d8ec3-37b2-4e38-b97e-8cd1e22ae1a1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:46.894340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4aa3f-79c6-43e6-8e73-0da37660d472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:46.895096 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4aa3f-79c6-43e6-8e73-0da37660d472 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:46.895363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4aa3f-79c6-43e6-8e73-0da37660d472 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:46.895622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4aa3f-79c6-43e6-8e73-0da37660d472 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:46.906399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4aa3f-79c6-43e6-8e73-0da37660d472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-fbf4aa3f-79c6-43e6-8e73-0da37660d472 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:46.911237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:46.912087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:46.913037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:46.913037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:46.913037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:46.913037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:46.913037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:46.913037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:46.913037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:46.913037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:46.913037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:46.913037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:46.913037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:46.913037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:46.913037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 214/423] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:04:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:04:46.921625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4aa3f-79c6-43e6-8e73-0da37660d472 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:46.926250 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36dfe90-957e-4540-b32e-2ce79f4592e1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:46.971490 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:46.988470 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef9b9b-00fc-4309-8b32-756db0dadcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:46.989137 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef9b9b-00fc-4309-8b32-756db0dadcdf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:46.989409 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef9b9b-00fc-4309-8b32-756db0dadcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:46.989679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef9b9b-00fc-4309-8b32-756db0dadcdf None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:46.995300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef9b9b-00fc-4309-8b32-756db0dadcdf None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:46.997052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef9b9b-00fc-4309-8b32-756db0dadcdf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:47.005363 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 210/424] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:46 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:47.006394 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef9b9b-00fc-4309-8b32-756db0dadcdf None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:47.056103 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:47.117721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4aa3f-79c6-43e6-8e73-0da37660d472 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:47.159335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fdb2bc-842f-45bc-b586-3fe4af5c6486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:47.160425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fdb2bc-842f-45bc-b586-3fe4af5c6486 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:47.160922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fdb2bc-842f-45bc-b586-3fe4af5c6486 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:47.161451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fdb2bc-842f-45bc-b586-3fe4af5c6486 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:47.162334 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fdb2bc-842f-45bc-b586-3fe4af5c6486 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:47.162825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fdb2bc-842f-45bc-b586-3fe4af5c6486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:47.164540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fdb2bc-842f-45bc-b586-3fe4af5c6486 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:47.173867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 215/425] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:04:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:47.175475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707086a7-5eed-4423-802b-2a700befd807 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:47.175973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707086a7-5eed-4423-802b-2a700befd807 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:47.176459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707086a7-5eed-4423-802b-2a700befd807 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:47.182434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef9b9b-00fc-4309-8b32-756db0dadcdf None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:47.182820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 216/426] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:47.184132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707086a7-5eed-4423-802b-2a700befd807 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:47.221006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389909e-bb3d-48cf-81ef-7709b54dd700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:47.221948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389909e-bb3d-48cf-81ef-7709b54dd700 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:47.222162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389909e-bb3d-48cf-81ef-7709b54dd700 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:47.222426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389909e-bb3d-48cf-81ef-7709b54dd700 None admin] PATH_INFO: `/v3/groups/362a436695ce418b9853d511fd6b8243` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:47.223940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389909e-bb3d-48cf-81ef-7709b54dd700 None admin] RBAC: Authorizing `identity:get_group(group_id=362a436695ce418b9853d511fd6b8243)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:47.225769 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389909e-bb3d-48cf-81ef-7709b54dd700 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:47.227743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 211/427] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:47 2022] GET /identity/v3/groups/362a436695ce418b9853d511fd6b8243 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:47.230376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49fb08-ace7-4ced-b5d6-aa09cf1f8d7f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:47.230376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49fb08-ace7-4ced-b5d6-aa09cf1f8d7f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:47.230376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49fb08-ace7-4ced-b5d6-aa09cf1f8d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:47.251183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6e49fb08-ace7-4ced-b5d6-aa09cf1f8d7f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:47.252400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6e49fb08-ace7-4ced-b5d6-aa09cf1f8d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:47.304823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707086a7-5eed-4423-802b-2a700befd807 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:47.319233 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:47.336616 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 212/428] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:47.336616 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49fb08-ace7-4ced-b5d6-aa09cf1f8d7f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:47.345965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:47.359446 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888f621-deaf-4773-b527-16ae24ca8c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:47.360496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888f621-deaf-4773-b527-16ae24ca8c81 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:47.360987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888f621-deaf-4773-b527-16ae24ca8c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:47.361449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888f621-deaf-4773-b527-16ae24ca8c81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:47.442441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49fb08-ace7-4ced-b5d6-aa09cf1f8d7f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:47.480831 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5415c81-42e7-442c-bdfe-1f376fd07f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:47.481348 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5415c81-42e7-442c-bdfe-1f376fd07f92 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:47.481615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5415c81-42e7-442c-bdfe-1f376fd07f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:47.481876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5415c81-42e7-442c-bdfe-1f376fd07f92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:47.482707 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5415c81-42e7-442c-bdfe-1f376fd07f92 None admin] RBAC: Adding query filter params (group.id=362a436695ce418b9853d511fd6b8243, role.id=2921a7146fd9408396ca98a4be4ecd3f, scope.project.id=9edb599ff8bb4c67b467b712ea2dc700) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:47.483310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5415c81-42e7-442c-bdfe-1f376fd07f92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:47.484846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5415c81-42e7-442c-bdfe-1f376fd07f92 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:47.491424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 213/429] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Sep 23 15:04:47 2022] GET /identity/v3/role_assignments?group.id=362a436695ce418b9853d511fd6b8243&scope.project.id=9edb599ff8bb4c67b467b712ea2dc700&role.id=2921a7146fd9408396ca98a4be4ecd3f => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:47.499916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888f621-deaf-4773-b527-16ae24ca8c81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:47.501953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888f621-deaf-4773-b527-16ae24ca8c81 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:47.630025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 217/430] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:47.635485 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5415c81-42e7-442c-bdfe-1f376fd07f92 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:47.702861 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888f621-deaf-4773-b527-16ae24ca8c81 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:47.716676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:47.764859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5415c81-42e7-442c-bdfe-1f376fd07f92 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:47.768901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:47.810646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f512c22-d20e-4cac-b802-67139345a68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:47.811603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f512c22-d20e-4cac-b802-67139345a68d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:47.812042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f512c22-d20e-4cac-b802-67139345a68d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:47.812512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f512c22-d20e-4cac-b802-67139345a68d None admin] PATH_INFO: `/v3/projects/9edb599ff8bb4c67b467b712ea2dc700` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:47.814959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f512c22-d20e-4cac-b802-67139345a68d None admin] RBAC: Authorizing `identity:get_project(project_id=9edb599ff8bb4c67b467b712ea2dc700)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:47.816765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f512c22-d20e-4cac-b802-67139345a68d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:47.818767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 214/431] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:47 2022] GET /identity/v3/projects/9edb599ff8bb4c67b467b712ea2dc700 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:47.824714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f512c22-d20e-4cac-b802-67139345a68d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:47.872733 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888f621-deaf-4773-b527-16ae24ca8c81 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:47.919958 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54a095-e2c0-4b6c-a1dc-4fd3d10ead1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:47.919958 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54a095-e2c0-4b6c-a1dc-4fd3d10ead1e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:47.920257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54a095-e2c0-4b6c-a1dc-4fd3d10ead1e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:47.920335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54a095-e2c0-4b6c-a1dc-4fd3d10ead1e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:47.932033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54a095-e2c0-4b6c-a1dc-4fd3d10ead1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-8a54a095-e2c0-4b6c-a1dc-4fd3d10ead1e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:47.937530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:47.938389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:47.939243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:47.939243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:47.939243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:47.939243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:47.939243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:47.939243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:47.939243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:47.939243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:47.939243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:47.939243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:47.939243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 15:04:47.939243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:47.943608 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 218/432] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:04:47.944880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54a095-e2c0-4b6c-a1dc-4fd3d10ead1e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:47.956238 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f512c22-d20e-4cac-b802-67139345a68d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:47.986023 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:47.999072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12950a44-8604-4cc7-ba83-fc9424805e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:48.000113 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12950a44-8604-4cc7-ba83-fc9424805e80 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:48.000576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12950a44-8604-4cc7-ba83-fc9424805e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:48.001081 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12950a44-8604-4cc7-ba83-fc9424805e80 None admin] PATH_INFO: `/v3/groups/362a436695ce418b9853d511fd6b8243` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:48.004411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12950a44-8604-4cc7-ba83-fc9424805e80 None admin] RBAC: Authorizing `identity:get_group(group_id=362a436695ce418b9853d511fd6b8243)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:48.006540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12950a44-8604-4cc7-ba83-fc9424805e80 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:48.009318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 215/433] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:47 2022] GET /identity/v3/groups/362a436695ce418b9853d511fd6b8243 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:48.016473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12950a44-8604-4cc7-ba83-fc9424805e80 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:48.053330 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:48.067821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54a095-e2c0-4b6c-a1dc-4fd3d10ead1e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:48.113669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c350e5-610a-45ed-8463-f99a1cf54a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:48.114368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c350e5-610a-45ed-8463-f99a1cf54a7d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:48.114640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c350e5-610a-45ed-8463-f99a1cf54a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:48.114906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c350e5-610a-45ed-8463-f99a1cf54a7d None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:48.116218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c350e5-610a-45ed-8463-f99a1cf54a7d None admin] RBAC: Adding query filter params (name=member) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:48.116574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c350e5-610a-45ed-8463-f99a1cf54a7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:48.118005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c350e5-610a-45ed-8463-f99a1cf54a7d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:48.128382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 219/434] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:04:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:48.130127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70ed7f-c518-4274-8cdc-1bd4d2b82350 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:48.130372 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70ed7f-c518-4274-8cdc-1bd4d2b82350 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:48.130993 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70ed7f-c518-4274-8cdc-1bd4d2b82350 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:48.132008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 220/435] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:48.138295 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70ed7f-c518-4274-8cdc-1bd4d2b82350 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:48.161638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12950a44-8604-4cc7-ba83-fc9424805e80 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:48.205644 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6221a-2d39-4d06-ad17-f55eec3c4d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:48.206247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6221a-2d39-4d06-ad17-f55eec3c4d07 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:48.206632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6221a-2d39-4d06-ad17-f55eec3c4d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:48.206878 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6221a-2d39-4d06-ad17-f55eec3c4d07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:48.207594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6221a-2d39-4d06-ad17-f55eec3c4d07 None admin] RBAC: Adding query filter params (group.id=362a436695ce418b9853d511fd6b8243, role.id=704c67b455b54fb48089faef4aba8ccd, scope.project.id=9edb599ff8bb4c67b467b712ea2dc700) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:48.207901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6221a-2d39-4d06-ad17-f55eec3c4d07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:48.209356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6221a-2d39-4d06-ad17-f55eec3c4d07 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:48.220646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 216/436] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Sep 23 15:04:48 2022] GET /identity/v3/role_assignments?group.id=362a436695ce418b9853d511fd6b8243&scope.project.id=9edb599ff8bb4c67b467b712ea2dc700&role.id=704c67b455b54fb48089faef4aba8ccd => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:48.220646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b730635-34a7-4fa0-906a-32cdce4fb47a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:48.220646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b730635-34a7-4fa0-906a-32cdce4fb47a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:48.220646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b730635-34a7-4fa0-906a-32cdce4fb47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:48.240502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5b730635-34a7-4fa0-906a-32cdce4fb47a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:48.242413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5b730635-34a7-4fa0-906a-32cdce4fb47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:48.289353 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70ed7f-c518-4274-8cdc-1bd4d2b82350 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:48.336931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:48.343683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59139b43-3a85-45b6-b78e-32d52f077218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:48.344755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59139b43-3a85-45b6-b78e-32d52f077218 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:48.345263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59139b43-3a85-45b6-b78e-32d52f077218 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:48.345712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59139b43-3a85-45b6-b78e-32d52f077218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:48.357362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 217/437] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:48.372883 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:48.428944 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59139b43-3a85-45b6-b78e-32d52f077218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:48.430769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59139b43-3a85-45b6-b78e-32d52f077218 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:48.556663 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 221/438] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:48.563321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b730635-34a7-4fa0-906a-32cdce4fb47a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:48.648192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2d682-054b-4383-8b5b-b234ef370e8b None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:48.648933 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2d682-054b-4383-8b5b-b234ef370e8b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:48.649409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2d682-054b-4383-8b5b-b234ef370e8b None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:48.651190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 222/439] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:48.657605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa22e14-f378-46ec-bcad-60ffaf85ba22 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:48.658063 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa22e14-f378-46ec-bcad-60ffaf85ba22 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:48.658491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa22e14-f378-46ec-bcad-60ffaf85ba22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:48.673189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b730635-34a7-4fa0-906a-32cdce4fb47a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:48.680457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6fa22e14-f378-46ec-bcad-60ffaf85ba22 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:48.681741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6fa22e14-f378-46ec-bcad-60ffaf85ba22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:48.712989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f618b1-beb8-44f1-967c-6e17ed5d2b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:48.713643 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f618b1-beb8-44f1-967c-6e17ed5d2b86 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:48.713915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f618b1-beb8-44f1-967c-6e17ed5d2b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:48.714183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f618b1-beb8-44f1-967c-6e17ed5d2b86 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:48.715324 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f618b1-beb8-44f1-967c-6e17ed5d2b86 None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:48.716853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f618b1-beb8-44f1-967c-6e17ed5d2b86 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:48.718683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 218/440] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:48 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:48.719334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f618b1-beb8-44f1-967c-6e17ed5d2b86 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:48.735845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:48.762951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 223/441] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:48.764171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa22e14-f378-46ec-bcad-60ffaf85ba22 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:48.792710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:48.840435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f618b1-beb8-44f1-967c-6e17ed5d2b86 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:48.871653 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa22e14-f378-46ec-bcad-60ffaf85ba22 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:48.876259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cf5ab-a156-49ea-b095-762c670da804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:48.876910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cf5ab-a156-49ea-b095-762c670da804 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:48.877185 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cf5ab-a156-49ea-b095-762c670da804 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:48.877451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cf5ab-a156-49ea-b095-762c670da804 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:48.888236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cf5ab-a156-49ea-b095-762c670da804 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-f56cf5ab-a156-49ea-b095-762c670da804 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:48.893099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:48.893988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:48.894777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:48.894777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:48.894777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:48.894777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:48.894777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:48.894777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:48.894777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:48.894777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:48.894777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:48.894777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:48.894777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:48.894777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:48.894777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 219/442] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:04:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:48.900928 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cf5ab-a156-49ea-b095-762c670da804 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:48.909974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0406e6-6d04-462c-9930-7c4c84be7930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:48.910687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0406e6-6d04-462c-9930-7c4c84be7930 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:48.911034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0406e6-6d04-462c-9930-7c4c84be7930 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:48.911362 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0406e6-6d04-462c-9930-7c4c84be7930 None admin] PATH_INFO: `/v3/groups/362a436695ce418b9853d511fd6b8243` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:48.912547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0406e6-6d04-462c-9930-7c4c84be7930 None admin] RBAC: Authorizing `identity:get_group(group_id=362a436695ce418b9853d511fd6b8243)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:48.914043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0406e6-6d04-462c-9930-7c4c84be7930 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:48.916147 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 224/443] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:48 2022] GET /identity/v3/groups/362a436695ce418b9853d511fd6b8243 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:48.921621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0406e6-6d04-462c-9930-7c4c84be7930 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:48.971369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:49.017977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cf5ab-a156-49ea-b095-762c670da804 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:49.028247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0406e6-6d04-462c-9930-7c4c84be7930 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:49.033388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:49.054797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a15f8e-ff05-4774-bc8a-c96558b8bb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:49.055495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a15f8e-ff05-4774-bc8a-c96558b8bb07 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:49.055769 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a15f8e-ff05-4774-bc8a-c96558b8bb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:49.056066 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a15f8e-ff05-4774-bc8a-c96558b8bb07 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:49.056770 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a15f8e-ff05-4774-bc8a-c96558b8bb07 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:49.057069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a15f8e-ff05-4774-bc8a-c96558b8bb07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:49.058592 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a15f8e-ff05-4774-bc8a-c96558b8bb07 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:49.062184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb0625-a856-4154-9692-adedbae757ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:49.062807 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb0625-a856-4154-9692-adedbae757ce None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:49.063077 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb0625-a856-4154-9692-adedbae757ce None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:49.063342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb0625-a856-4154-9692-adedbae757ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:49.064085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb0625-a856-4154-9692-adedbae757ce None admin] RBAC: Adding query filter params (group.id=362a436695ce418b9853d511fd6b8243, role.id=2921a7146fd9408396ca98a4be4ecd3f, scope.project.id=117d6fb504534ef2bd643528f241dffa) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:49.064398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb0625-a856-4154-9692-adedbae757ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:49.065887 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb0625-a856-4154-9692-adedbae757ce None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:49.066644 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 220/444] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:04:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:49.067447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a15f8e-ff05-4774-bc8a-c96558b8bb07 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:49.076555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 225/445] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Sep 23 15:04:48 2022] GET /identity/v3/role_assignments?group.id=362a436695ce418b9853d511fd6b8243&scope.project.id=117d6fb504534ef2bd643528f241dffa&role.id=2921a7146fd9408396ca98a4be4ecd3f => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:49.078591 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb0625-a856-4154-9692-adedbae757ce None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:49.181373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a15f8e-ff05-4774-bc8a-c96558b8bb07 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:49.203668 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb0625-a856-4154-9692-adedbae757ce None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:49.218984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:49.223495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6ba67-4dfd-4ede-a60b-1a3717463241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:49.224341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6ba67-4dfd-4ede-a60b-1a3717463241 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:49.224791 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6ba67-4dfd-4ede-a60b-1a3717463241 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:49.225033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6ba67-4dfd-4ede-a60b-1a3717463241 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:49.237973 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6ba67-4dfd-4ede-a60b-1a3717463241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-bfa6ba67-4dfd-4ede-a60b-1a3717463241 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:49.244474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:49.245440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:49.246311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:49.246311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:49.246311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:49.246311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:49.246311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:49.246311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:49.246311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:49.246311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:49.246311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:49.246311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:49.246311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:04:49.246311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:49.246311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 221/446] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:04:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:49.249128 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8cfe4c-e249-429c-a616-4a2ee8f16e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:49.249806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cfe4c-e249-429c-a616-4a2ee8f16e0e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:49.250077 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cfe4c-e249-429c-a616-4a2ee8f16e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:49.250339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cfe4c-e249-429c-a616-4a2ee8f16e0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:49.256065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6ba67-4dfd-4ede-a60b-1a3717463241 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:49.297747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:49.332656 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8cfe4c-e249-429c-a616-4a2ee8f16e0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:49.333661 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8cfe4c-e249-429c-a616-4a2ee8f16e0e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:49.377174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6ba67-4dfd-4ede-a60b-1a3717463241 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:49.419565 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f63cfe-bf1e-431c-a343-5c39f950bcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:49.420212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f63cfe-bf1e-431c-a343-5c39f950bcab None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:49.420477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f63cfe-bf1e-431c-a343-5c39f950bcab None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:49.420765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f63cfe-bf1e-431c-a343-5c39f950bcab None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:49.421439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f63cfe-bf1e-431c-a343-5c39f950bcab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:49.421724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f63cfe-bf1e-431c-a343-5c39f950bcab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:49.423551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f63cfe-bf1e-431c-a343-5c39f950bcab None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:49.433218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 222/447] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:49.439142 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f63cfe-bf1e-431c-a343-5c39f950bcab None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:49.461205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 226/448] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:49.467806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8cfe4c-e249-429c-a616-4a2ee8f16e0e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:49.482183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:49.547331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:49.551425 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f63cfe-bf1e-431c-a343-5c39f950bcab None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:49.576796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8cfe4c-e249-429c-a616-4a2ee8f16e0e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:49.589734 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346c5f2-11cb-4fa5-91d8-1413b76f52cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:49.590430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346c5f2-11cb-4fa5-91d8-1413b76f52cc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:49.590820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346c5f2-11cb-4fa5-91d8-1413b76f52cc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:49.590992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346c5f2-11cb-4fa5-91d8-1413b76f52cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:49.611435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527dd69b-cf37-46a9-a568-69daa7f0acf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:49.612145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dd69b-cf37-46a9-a568-69daa7f0acf3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:49.612472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dd69b-cf37-46a9-a568-69daa7f0acf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:49.612839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527dd69b-cf37-46a9-a568-69daa7f0acf3 None admin] PATH_INFO: `/v3/projects/117d6fb504534ef2bd643528f241dffa` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:49.613953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527dd69b-cf37-46a9-a568-69daa7f0acf3 None admin] RBAC: Authorizing `identity:get_project(project_id=117d6fb504534ef2bd643528f241dffa)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:49.615406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527dd69b-cf37-46a9-a568-69daa7f0acf3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:49.617218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 227/449] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:49 2022] GET /identity/v3/projects/117d6fb504534ef2bd643528f241dffa => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:49.622709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527dd69b-cf37-46a9-a568-69daa7f0acf3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:49.659516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346c5f2-11cb-4fa5-91d8-1413b76f52cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:49.661001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346c5f2-11cb-4fa5-91d8-1413b76f52cc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:49.705183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:49.725197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527dd69b-cf37-46a9-a568-69daa7f0acf3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:49.754328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:49.759703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 223/450] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 15:04:49.761291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79635681-4a23-4792-98cd-1c64ea5530fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:49.761935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79635681-4a23-4792-98cd-1c64ea5530fb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:49.762209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79635681-4a23-4792-98cd-1c64ea5530fb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:49.762473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79635681-4a23-4792-98cd-1c64ea5530fb None admin] PATH_INFO: `/v3/groups/362a436695ce418b9853d511fd6b8243` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:49.763611 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79635681-4a23-4792-98cd-1c64ea5530fb None admin] RBAC: Authorizing `identity:get_group(group_id=362a436695ce418b9853d511fd6b8243)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:49.765047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346c5f2-11cb-4fa5-91d8-1413b76f52cc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:49.765170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79635681-4a23-4792-98cd-1c64ea5530fb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:49.766672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 228/451] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:49 2022] GET /identity/v3/groups/362a436695ce418b9853d511fd6b8243 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:49.772155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79635681-4a23-4792-98cd-1c64ea5530fb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:49.867084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346c5f2-11cb-4fa5-91d8-1413b76f52cc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:49.876552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79635681-4a23-4792-98cd-1c64ea5530fb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:49.899505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b402b67-9abb-4f06-b93c-f202ced60a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:49.900171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b402b67-9abb-4f06-b93c-f202ced60a92 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:49.900471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b402b67-9abb-4f06-b93c-f202ced60a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:49.900852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b402b67-9abb-4f06-b93c-f202ced60a92 None admin] PATH_INFO: `/v3/projects/a6e3ae74e78241ac8bfc0ae2c7103bec` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:49.901993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b402b67-9abb-4f06-b93c-f202ced60a92 None admin] RBAC: Authorizing `identity:get_project(project_id=a6e3ae74e78241ac8bfc0ae2c7103bec)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:49.903435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b402b67-9abb-4f06-b93c-f202ced60a92 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:49.905109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 224/452] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:04:49 2022] GET /identity/v3/projects/a6e3ae74e78241ac8bfc0ae2c7103bec => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:49.910705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b402b67-9abb-4f06-b93c-f202ced60a92 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:49.912787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f55679-e156-4aa9-af2f-32f7680073b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:49.913394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f55679-e156-4aa9-af2f-32f7680073b9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:49.913663 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f55679-e156-4aa9-af2f-32f7680073b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:49.913926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f55679-e156-4aa9-af2f-32f7680073b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:49.914622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f55679-e156-4aa9-af2f-32f7680073b9 None admin] RBAC: Adding query filter params (group.id=362a436695ce418b9853d511fd6b8243, role.id=704c67b455b54fb48089faef4aba8ccd, scope.project.id=117d6fb504534ef2bd643528f241dffa) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:49.914922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f55679-e156-4aa9-af2f-32f7680073b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:49.916899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f55679-e156-4aa9-af2f-32f7680073b9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:49.922760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 229/453] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Sep 23 15:04:49 2022] GET /identity/v3/role_assignments?group.id=362a436695ce418b9853d511fd6b8243&scope.project.id=117d6fb504534ef2bd643528f241dffa&role.id=704c67b455b54fb48089faef4aba8ccd => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:49.984292 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:50.020631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b402b67-9abb-4f06-b93c-f202ced60a92 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:50.055814 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ce9d5-a6c4-4bc1-8bf5-afa66b6e35bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:50.056481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ce9d5-a6c4-4bc1-8bf5-afa66b6e35bf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:50.056783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ce9d5-a6c4-4bc1-8bf5-afa66b6e35bf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:50.057048 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ce9d5-a6c4-4bc1-8bf5-afa66b6e35bf None admin] PATH_INFO: `/v3/groups/3a0dd69a67f24a65ad58847e94c8c867` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:50.058199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ce9d5-a6c4-4bc1-8bf5-afa66b6e35bf None admin] RBAC: Authorizing `identity:get_group(group_id=3a0dd69a67f24a65ad58847e94c8c867)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:50.059680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ce9d5-a6c4-4bc1-8bf5-afa66b6e35bf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:50.061426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 225/454] 10.222.0.75 () {56 vars in 1180 bytes} [Fri Sep 23 15:04:49 2022] GET /identity/v3/groups/3a0dd69a67f24a65ad58847e94c8c867 => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:50.067587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f55679-e156-4aa9-af2f-32f7680073b9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:50.085161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:50.180352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f55679-e156-4aa9-af2f-32f7680073b9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:50.215879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e59aa5-5d34-4ed0-baec-72d3f4264eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:50.216544 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e59aa5-5d34-4ed0-baec-72d3f4264eea None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:50.216854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e59aa5-5d34-4ed0-baec-72d3f4264eea None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:50.217123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e59aa5-5d34-4ed0-baec-72d3f4264eea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:50.217859 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e59aa5-5d34-4ed0-baec-72d3f4264eea None admin] RBAC: Adding query filter params (group.id=3a0dd69a67f24a65ad58847e94c8c867, role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=a6e3ae74e78241ac8bfc0ae2c7103bec) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:50.222393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e59aa5-5d34-4ed0-baec-72d3f4264eea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:50.222393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e59aa5-5d34-4ed0-baec-72d3f4264eea None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:50.225721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 230/455] 10.222.0.75 () {56 vars in 1376 bytes} [Fri Sep 23 15:04:50 2022] GET /identity/v3/role_assignments?group.id=3a0dd69a67f24a65ad58847e94c8c867&scope.project.id=a6e3ae74e78241ac8bfc0ae2c7103bec&role.id=ae444a489623434999d758b71ae2dc87 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:52.960367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235612c9-c0bf-4b39-beba-a780c5926dd7 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:52.960957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235612c9-c0bf-4b39-beba-a780c5926dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:52.960957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235612c9-c0bf-4b39-beba-a780c5926dd7 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:52.962773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 226/456] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:52.968031 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c72d2f-3f90-40f0-a0cf-1ae5f0d45ea1 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:52.968892 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c72d2f-3f90-40f0-a0cf-1ae5f0d45ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:52.969377 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c72d2f-3f90-40f0-a0cf-1ae5f0d45ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:52.998232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-90c72d2f-3f90-40f0-a0cf-1ae5f0d45ea1 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:52.999737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-90c72d2f-3f90-40f0-a0cf-1ae5f0d45ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:53.011837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1b06a-fee0-4a69-b8e6-3a860869980f None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:53.012416 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1b06a-fee0-4a69-b8e6-3a860869980f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:53.012927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1b06a-fee0-4a69-b8e6-3a860869980f None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:53.014740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 227/457] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:53.020170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8dc40-a44d-4fad-bf94-e38f0e638cf0 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:53.020764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8dc40-a44d-4fad-bf94-e38f0e638cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:53.021264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8dc40-a44d-4fad-bf94-e38f0e638cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:53.057530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-caa8dc40-a44d-4fad-bf94-e38f0e638cf0 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:53.058735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-caa8dc40-a44d-4fad-bf94-e38f0e638cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:53.088175 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 231/458] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:53.140088 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 228/459] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:53.379492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c72d2f-3f90-40f0-a0cf-1ae5f0d45ea1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:53.388749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:53.419188 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25f841-c141-439f-922e-ee005d049342 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:53.419507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25f841-c141-439f-922e-ee005d049342 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:53.419744 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25f841-c141-439f-922e-ee005d049342 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:53.421215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 229/460] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:04:53.435775 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9341aa9-7d01-486c-b91a-dba956a2648e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:53.436109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9341aa9-7d01-486c-b91a-dba956a2648e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:53.436312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9341aa9-7d01-486c-b91a-dba956a2648e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:53.478226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f9341aa9-7d01-486c-b91a-dba956a2648e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:53.479492 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f9341aa9-7d01-486c-b91a-dba956a2648e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:53.516663 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c72d2f-3f90-40f0-a0cf-1ae5f0d45ea1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:53.566531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:53.573573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602dd16-beb8-4ad1-b4d2-c1550bbd099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:53.574430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602dd16-beb8-4ad1-b4d2-c1550bbd099b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:53.574930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602dd16-beb8-4ad1-b4d2-c1550bbd099b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:53.575459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602dd16-beb8-4ad1-b4d2-c1550bbd099b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:53.577506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 230/461] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:53.578711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9341aa9-7d01-486c-b91a-dba956a2648e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:53.592154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602dd16-beb8-4ad1-b4d2-c1550bbd099b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:53.593685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602dd16-beb8-4ad1-b4d2-c1550bbd099b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-b602dd16-beb8-4ad1-b4d2-c1550bbd099b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:04:53.613583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:53.614542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:04:53.615444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:04:53.615444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:53.615444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:53.615444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 232/462] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:53.627815 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602dd16-beb8-4ad1-b4d2-c1550bbd099b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:53.762895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9341aa9-7d01-486c-b91a-dba956a2648e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:53.770052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:53.821163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602dd16-beb8-4ad1-b4d2-c1550bbd099b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:53.862618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb4104-1968-42af-a3c6-9c190ed1a2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:53.864217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb4104-1968-42af-a3c6-9c190ed1a2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:53.865392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb4104-1968-42af-a3c6-9c190ed1a2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:53.865864 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb4104-1968-42af-a3c6-9c190ed1a2e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:53.867073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb4104-1968-42af-a3c6-9c190ed1a2e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:53.868566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb4104-1968-42af-a3c6-9c190ed1a2e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:53.870191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb4104-1968-42af-a3c6-9c190ed1a2e5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:53.873651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e916b-a1d2-43f1-b7f1-e6dda6768002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:53.874604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e916b-a1d2-43f1-b7f1-e6dda6768002 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:53.875038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e916b-a1d2-43f1-b7f1-e6dda6768002 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:53.875852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e916b-a1d2-43f1-b7f1-e6dda6768002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:53.899301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e916b-a1d2-43f1-b7f1-e6dda6768002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:53.902377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e916b-a1d2-43f1-b7f1-e6dda6768002 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:53.912224 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 233/463] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:53.917985 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb4104-1968-42af-a3c6-9c190ed1a2e5 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-e34e916b-a1d2-43f1-b7f1-e6dda6768002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:04:53.927094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:53.928244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:04:53.929184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:04:53.929184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:53.929184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:53.929184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 231/464] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:53.929400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e916b-a1d2-43f1-b7f1-e6dda6768002 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:53.993832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:54.048919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb4104-1968-42af-a3c6-9c190ed1a2e5 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:54.069658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e916b-a1d2-43f1-b7f1-e6dda6768002 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:54.071606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:54.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2348e1-6c72-430a-9da1-7dafc1a4c05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:54.094728 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2348e1-6c72-430a-9da1-7dafc1a4c05a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:54.095123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2348e1-6c72-430a-9da1-7dafc1a4c05a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:54.096273 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2348e1-6c72-430a-9da1-7dafc1a4c05a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:54.109882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2348e1-6c72-430a-9da1-7dafc1a4c05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:54.111706 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2348e1-6c72-430a-9da1-7dafc1a4c05a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:54.115394 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b02a09-c4e9-4c25-bf4e-4023b1ae98d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:54.116154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02a09-c4e9-4c25-bf4e-4023b1ae98d2 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:54.116417 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02a09-c4e9-4c25-bf4e-4023b1ae98d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:54.116707 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02a09-c4e9-4c25-bf4e-4023b1ae98d2 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:54.117361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b02a09-c4e9-4c25-bf4e-4023b1ae98d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:54.119000 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b02a09-c4e9-4c25-bf4e-4023b1ae98d2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-6f2348e1-6c72-430a-9da1-7dafc1a4c05a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:04:54.129761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:54.131880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:04:54.133792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:04:54.133792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:54.133792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:54.133792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 234/465] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:04:54.133792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2348e1-6c72-430a-9da1-7dafc1a4c05a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:54.152703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 232/466] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:04:53 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:04:54.166510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d97a62-3aef-4a85-9aed-405df1fcdde6 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:54.166833 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d97a62-3aef-4a85-9aed-405df1fcdde6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:54.167074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d97a62-3aef-4a85-9aed-405df1fcdde6 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:54.168529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 233/467] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:54 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:54.169296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d97a62-3aef-4a85-9aed-405df1fcdde6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:54.278893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2348e1-6c72-430a-9da1-7dafc1a4c05a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:54.299596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d97a62-3aef-4a85-9aed-405df1fcdde6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:54.301735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:54.329288 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b7296-fa09-4921-afc9-3c0abdef9f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:54.329965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b7296-fa09-4921-afc9-3c0abdef9f5b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:54.330376 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b7296-fa09-4921-afc9-3c0abdef9f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:54.330563 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b7296-fa09-4921-afc9-3c0abdef9f5b None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:54.331223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b7296-fa09-4921-afc9-3c0abdef9f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:54.331532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b7296-fa09-4921-afc9-3c0abdef9f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:54.333306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b7296-fa09-4921-afc9-3c0abdef9f5b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:54.346264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be9b28-ed16-43e0-9f70-4bc58efda170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:54.347232 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be9b28-ed16-43e0-9f70-4bc58efda170 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:54.347773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be9b28-ed16-43e0-9f70-4bc58efda170 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:54.347994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 235/468] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:54.352491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77f169-beb8-407a-b3ee-f20fad2df351 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:54.352491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77f169-beb8-407a-b3ee-f20fad2df351 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:54.352491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77f169-beb8-407a-b3ee-f20fad2df351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:54.353575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be9b28-ed16-43e0-9f70-4bc58efda170 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:54.354693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be9b28-ed16-43e0-9f70-4bc58efda170 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:54.354990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be9b28-ed16-43e0-9f70-4bc58efda170 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:54.356376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be9b28-ed16-43e0-9f70-4bc58efda170 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:54.370514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 234/469] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:54.371176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be9b28-ed16-43e0-9f70-4bc58efda170 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:54.373965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-dc77f169-beb8-407a-b3ee-f20fad2df351 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:54.375950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dc77f169-beb8-407a-b3ee-f20fad2df351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:54.405671 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:54.480629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 236/470] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:54.480629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77f169-beb8-407a-b3ee-f20fad2df351 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:54.540419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be9b28-ed16-43e0-9f70-4bc58efda170 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:54.591452 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6dfe5-a597-45f0-8b91-f26bef3ed4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:54.592034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77f169-beb8-407a-b3ee-f20fad2df351 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:54.592144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6dfe5-a597-45f0-8b91-f26bef3ed4fb None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:54.592400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6dfe5-a597-45f0-8b91-f26bef3ed4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:54.592698 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6dfe5-a597-45f0-8b91-f26bef3ed4fb None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:54.593546 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6dfe5-a597-45f0-8b91-f26bef3ed4fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:54.595135 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6dfe5-a597-45f0-8b91-f26bef3ed4fb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:54.621756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 235/471] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:04:54 2022] POST /identity/v3/users => generated 246 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:04:54.623511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8aac5-fad5-4c80-869d-eaa853fb09bc None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:54.623794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8aac5-fad5-4c80-869d-eaa853fb09bc None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:54.624172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8aac5-fad5-4c80-869d-eaa853fb09bc None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:54.625656 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 236/472] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:54.632204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5726bdc-e0a4-4e04-96e3-bb775295f604 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:54.632573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5726bdc-e0a4-4e04-96e3-bb775295f604 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:54.632889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5726bdc-e0a4-4e04-96e3-bb775295f604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:54.653488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075cad8-4287-4742-abdc-0db9abda141d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:54.654120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075cad8-4287-4742-abdc-0db9abda141d None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:54.654390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075cad8-4287-4742-abdc-0db9abda141d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:54.654657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075cad8-4287-4742-abdc-0db9abda141d None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:54.655318 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075cad8-4287-4742-abdc-0db9abda141d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:54.656575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a5726bdc-e0a4-4e04-96e3-bb775295f604 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:54.656895 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075cad8-4287-4742-abdc-0db9abda141d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:54.657874 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a5726bdc-e0a4-4e04-96e3-bb775295f604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:54.676061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:54.699587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 237/473] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:04:54 2022] POST /identity/v3/users => generated 248 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:04:54.778781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 237/474] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:04:54.798255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075cad8-4287-4742-abdc-0db9abda141d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:54.829296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:54.938512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075cad8-4287-4742-abdc-0db9abda141d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:54.978053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa7dcc-bf06-4446-8968-2b9238c59884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:54.978694 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa7dcc-bf06-4446-8968-2b9238c59884 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:54.978963 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa7dcc-bf06-4446-8968-2b9238c59884 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:54.979224 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa7dcc-bf06-4446-8968-2b9238c59884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:54.991235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa7dcc-bf06-4446-8968-2b9238c59884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:54.992749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa7dcc-bf06-4446-8968-2b9238c59884 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-6ffa7dcc-bf06-4446-8968-2b9238c59884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:04:55.007574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:55.008695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:04:55.010020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:04:55.010020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:55.010020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:55.010020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 238/475] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:55.016395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5726bdc-e0a4-4e04-96e3-bb775295f604 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:55.081574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa7dcc-bf06-4446-8968-2b9238c59884 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:55.132270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5726bdc-e0a4-4e04-96e3-bb775295f604 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:55.176468 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09566b20-8ce5-4234-83bc-4144c77b7ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:55.178922 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09566b20-8ce5-4234-83bc-4144c77b7ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:55.179566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09566b20-8ce5-4234-83bc-4144c77b7ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:55.181324 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09566b20-8ce5-4234-83bc-4144c77b7ed5 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:55.182887 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09566b20-8ce5-4234-83bc-4144c77b7ed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:55.183971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09566b20-8ce5-4234-83bc-4144c77b7ed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:55.185856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09566b20-8ce5-4234-83bc-4144c77b7ed5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:55.190351 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:55.212224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 238/476] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:55.222061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09566b20-8ce5-4234-83bc-4144c77b7ed5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:55.249110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:55.273872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa7dcc-bf06-4446-8968-2b9238c59884 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:55.349952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb72d06-5e97-489f-b351-bfa8156d5adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:55.350586 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb72d06-5e97-489f-b351-bfa8156d5adb None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:55.350839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb72d06-5e97-489f-b351-bfa8156d5adb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:55.351107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb72d06-5e97-489f-b351-bfa8156d5adb None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:55.351906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb72d06-5e97-489f-b351-bfa8156d5adb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:55.353668 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb72d06-5e97-489f-b351-bfa8156d5adb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:55.359879 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09566b20-8ce5-4234-83bc-4144c77b7ed5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:55.381398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.sql.core [None req-ebb72d06-5e97-489f-b351-bfa8156d5adb None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 23 15:04:55.381398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 23 15:04:55.381398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [parameters: {'id': 'e489f5f2ed044d1aa11f853a0d7fbac2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 23 15:04:55.381398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81864) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 23 15:04:55.396080 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d59f4-1325-4962-a674-ed6858d59b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:55.397145 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d59f4-1325-4962-a674-ed6858d59b65 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:55.397638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d59f4-1325-4962-a674-ed6858d59b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:55.398114 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d59f4-1325-4962-a674-ed6858d59b65 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:55.399025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d59f4-1325-4962-a674-ed6858d59b65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:55.400909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d59f4-1325-4962-a674-ed6858d59b65 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-ebb72d06-5e97-489f-b351-bfa8156d5adb None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application conn.query(q) Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application result.read() Sep 23 15:04:55.410066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return ref.to_dict() Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 15:04:55.410996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application next(self.gen) Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application yield resource Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application next(self.gen) Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self.session.rollback() Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self.force_reraise() Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise self.value Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application session.commit() Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 23 15:04:55.412472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._prepare_impl() Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self.session.flush() Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._flush(objects) Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application compat.raise_( Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application flush_context.execute() Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rec.execute(self) Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 23 15:04:55.413423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 15:04:55.414275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application conn.query(q) Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application result.read() Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 15:04:55.416073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [parameters: {'id': 'e489f5f2ed044d1aa11f853a0d7fbac2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:55.418454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 23 15:04:55.420566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:55.422585 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 239/477] 10.222.0.75 () {60 vars in 1131 bytes} [Fri Sep 23 15:04:55 2022] POST /identity/v3/roles => generated 144 bytes in 330 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:55.422585 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb72d06-5e97-489f-b351-bfa8156d5adb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:55.450670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 239/478] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:04:55 2022] POST /identity/v3/users => generated 249 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:04:55.582574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb72d06-5e97-489f-b351-bfa8156d5adb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:55.632229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:55.640936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89dc993-2e82-4d3b-8e0d-0bf4588a44a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:55.641771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dc993-2e82-4d3b-8e0d-0bf4588a44a9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:55.642220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dc993-2e82-4d3b-8e0d-0bf4588a44a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:55.642680 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dc993-2e82-4d3b-8e0d-0bf4588a44a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:55.660263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89dc993-2e82-4d3b-8e0d-0bf4588a44a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d89dc993-2e82-4d3b-8e0d-0bf4588a44a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:55.665796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:55.667231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:55.669210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:55.669210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:55.669210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:55.669210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:55.669210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:55.669210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:55.669210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:55.669210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:55.669210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:55.669210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:55.669210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 15:04:55.669210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:55.669210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 240/479] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:04:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:55.673520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d59f4-1325-4962-a674-ed6858d59b65 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:55.754743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:55.795624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d59f4-1325-4962-a674-ed6858d59b65 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:55.829418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f6bb2-b470-452e-8b19-3dc582b728ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:55.830021 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f6bb2-b470-452e-8b19-3dc582b728ad None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:55.830288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f6bb2-b470-452e-8b19-3dc582b728ad None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:55.830554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f6bb2-b470-452e-8b19-3dc582b728ad None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:55.831219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f6bb2-b470-452e-8b19-3dc582b728ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:55.831513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f6bb2-b470-452e-8b19-3dc582b728ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:55.832904 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f6bb2-b470-452e-8b19-3dc582b728ad None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:55.841828 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 240/480] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:04:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:56.764957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f2635-efbb-4833-8213-9b520d2d6a52 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:56.765471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f2635-efbb-4833-8213-9b520d2d6a52 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:56.765471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8f2635-efbb-4833-8213-9b520d2d6a52 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:56.767066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 241/481] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:56.782225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf6233-09d5-430d-b720-4b4d30792333 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:56.782722 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf6233-09d5-430d-b720-4b4d30792333 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:56.782796 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf6233-09d5-430d-b720-4b4d30792333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:56.805431 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0acf6233-09d5-430d-b720-4b4d30792333 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:56.806672 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0acf6233-09d5-430d-b720-4b4d30792333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:56.897829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 241/482] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:57.268905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f2635-efbb-4833-8213-9b520d2d6a52 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:57.394507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8f2635-efbb-4833-8213-9b520d2d6a52 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:57.429752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42397c2e-785b-48f7-85d6-d4efc9109ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:57.430387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42397c2e-785b-48f7-85d6-d4efc9109ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:57.430663 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42397c2e-785b-48f7-85d6-d4efc9109ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:57.431046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42397c2e-785b-48f7-85d6-d4efc9109ea4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:57.442273 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42397c2e-785b-48f7-85d6-d4efc9109ea4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-42397c2e-785b-48f7-85d6-d4efc9109ea4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:57.447178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:57.448210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:57.449033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:57.449033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:57.449033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:57.449033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:57.449033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:57.449033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:57.449033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:57.449033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:57.449033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:57.449033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:57.449033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:04:57.449033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:57.449033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 242/483] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:04:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:57.457917 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf6233-09d5-430d-b720-4b4d30792333 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:57.518247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec234eb-5d55-4088-8b45-ed38286cb56c None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:57.518597 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec234eb-5d55-4088-8b45-ed38286cb56c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:57.518815 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec234eb-5d55-4088-8b45-ed38286cb56c None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:57.520633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 243/484] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:57.529256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0487b-01d3-47ac-a9a8-f130995c03fa None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:57.529766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0487b-01d3-47ac-a9a8-f130995c03fa None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:57.530186 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0487b-01d3-47ac-a9a8-f130995c03fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:57.565992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:57.570786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-98e0487b-01d3-47ac-a9a8-f130995c03fa None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:57.572075 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-98e0487b-01d3-47ac-a9a8-f130995c03fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:57.633025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf6233-09d5-430d-b720-4b4d30792333 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:57.659339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:57.675793 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5a98e7-2305-4243-9317-292710579634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:57.676877 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a98e7-2305-4243-9317-292710579634 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:57.677695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a98e7-2305-4243-9317-292710579634 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:57.678323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a98e7-2305-4243-9317-292710579634 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:57.679196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a98e7-2305-4243-9317-292710579634 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:57.679626 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a98e7-2305-4243-9317-292710579634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:57.681541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5a98e7-2305-4243-9317-292710579634 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:57.681819 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 244/485] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:57.683288 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183c1e6-bf4b-4996-a09e-a1fc065e68c7 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:57.683561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183c1e6-bf4b-4996-a09e-a1fc065e68c7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:57.689104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183c1e6-bf4b-4996-a09e-a1fc065e68c7 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:57.690677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 245/486] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:57.691554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 242/487] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:04:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:57.697740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43b017-4af5-47b7-8313-88479f55f22d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:57.698798 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183c1e6-bf4b-4996-a09e-a1fc065e68c7 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:57.699648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43b017-4af5-47b7-8313-88479f55f22d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:57.700526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43b017-4af5-47b7-8313-88479f55f22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:57.737611 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-bf43b017-4af5-47b7-8313-88479f55f22d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:57.738773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-bf43b017-4af5-47b7-8313-88479f55f22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:57.812775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183c1e6-bf4b-4996-a09e-a1fc065e68c7 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:57.848740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3eb05f-2a6d-440f-9d3e-3d300f853c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:57.849774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eb05f-2a6d-440f-9d3e-3d300f853c12 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:57.850232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eb05f-2a6d-440f-9d3e-3d300f853c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:57.850716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eb05f-2a6d-440f-9d3e-3d300f853c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:57.862155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3eb05f-2a6d-440f-9d3e-3d300f853c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:57.863903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3eb05f-2a6d-440f-9d3e-3d300f853c12 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-3b3eb05f-2a6d-440f-9d3e-3d300f853c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:04:57.879894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:57.881002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:04:57.882068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:04:57.882068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:57.882068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:57.883716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 246/488] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:04:57.889371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3eb05f-2a6d-440f-9d3e-3d300f853c12 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:57.890099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 243/489] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:57.924828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:57.992193 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3eb05f-2a6d-440f-9d3e-3d300f853c12 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:57.997696 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43b017-4af5-47b7-8313-88479f55f22d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:58.026842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7cb2f8-27c9-4595-ad1c-12d6df888def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:58.027702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cb2f8-27c9-4595-ad1c-12d6df888def None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.028111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cb2f8-27c9-4595-ad1c-12d6df888def None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.028509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7cb2f8-27c9-4595-ad1c-12d6df888def None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.029311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7cb2f8-27c9-4595-ad1c-12d6df888def None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:58.029777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7cb2f8-27c9-4595-ad1c-12d6df888def None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:58.031272 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7cb2f8-27c9-4595-ad1c-12d6df888def None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:58.048976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 247/490] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:58.050407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7cb2f8-27c9-4595-ad1c-12d6df888def None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:58.093246 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:58.156438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7cb2f8-27c9-4595-ad1c-12d6df888def None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:58.164801 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:58.172614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43b017-4af5-47b7-8313-88479f55f22d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:58.193239 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb9b21-9b36-4c25-8864-5143cad5449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:58.194145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb9b21-9b36-4c25-8864-5143cad5449c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.194578 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb9b21-9b36-4c25-8864-5143cad5449c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.195071 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb9b21-9b36-4c25-8864-5143cad5449c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.203244 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb9b21-9b36-4c25-8864-5143cad5449c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:58.205024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb9b21-9b36-4c25-8864-5143cad5449c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7cfb9b21-9b36-4c25-8864-5143cad5449c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:04:58.224996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:58.225935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:58.226872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:58.226872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:58.226872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:04:58.226872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:04:58.226872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:04:58.226872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:04:58.226872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:04:58.226872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:04:58.226872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 23 15:04:58.226872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:58.227479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 248/491] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:04:58.234545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70528463-2ca5-4420-a64f-a2cf9443dbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:58.235508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70528463-2ca5-4420-a64f-a2cf9443dbda None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.235915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70528463-2ca5-4420-a64f-a2cf9443dbda None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.236310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70528463-2ca5-4420-a64f-a2cf9443dbda None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.237583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb9b21-9b36-4c25-8864-5143cad5449c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:58.252642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70528463-2ca5-4420-a64f-a2cf9443dbda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-70528463-2ca5-4420-a64f-a2cf9443dbda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:58.260015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:58.261115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:58.261987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:58.261987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:58.261987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:58.261987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:58.261987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:58.261987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:58.261987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:58.261987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:58.261987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:58.261987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:58.261987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:04:58.261987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:58.262984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 244/492] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:04:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:58.268983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70528463-2ca5-4420-a64f-a2cf9443dbda None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:58.372254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb9b21-9b36-4c25-8864-5143cad5449c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:58.389600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70528463-2ca5-4420-a64f-a2cf9443dbda None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:58.399688 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:58.413528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85afea4-2f2d-48bf-a969-13e683493112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:58.414513 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85afea4-2f2d-48bf-a969-13e683493112 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.414952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85afea4-2f2d-48bf-a969-13e683493112 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.415488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85afea4-2f2d-48bf-a969-13e683493112 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.416660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85afea4-2f2d-48bf-a969-13e683493112 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:58.417140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85afea4-2f2d-48bf-a969-13e683493112 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:58.418950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85afea4-2f2d-48bf-a969-13e683493112 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:58.426610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c15e8-a968-47ab-8581-15f4beb16cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:58.427249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c15e8-a968-47ab-8581-15f4beb16cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.427522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c15e8-a968-47ab-8581-15f4beb16cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.427788 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c15e8-a968-47ab-8581-15f4beb16cf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.428499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c15e8-a968-47ab-8581-15f4beb16cf5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:58.428856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c15e8-a968-47ab-8581-15f4beb16cf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:58.430299 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531c15e8-a968-47ab-8581-15f4beb16cf5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:58.445789 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 245/493] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:04:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:04:58.445789 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c15e8-a968-47ab-8581-15f4beb16cf5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:58.471919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 249/494] 10.222.0.75 () {56 vars in 1137 bytes} [Fri Sep 23 15:04:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:58.473296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85afea4-2f2d-48bf-a969-13e683493112 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:58.489844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:58.566915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c15e8-a968-47ab-8581-15f4beb16cf5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:58.582235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85afea4-2f2d-48bf-a969-13e683493112 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:58.599546 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a5872-0898-458e-ab1a-98322c271f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:58.600169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a5872-0898-458e-ab1a-98322c271f87 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.600439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a5872-0898-458e-ab1a-98322c271f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.600753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a5872-0898-458e-ab1a-98322c271f87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.611246 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a5872-0898-458e-ab1a-98322c271f87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:58.615679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043db32c-f4ea-45d1-ace4-fd54aa3cc0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:58.616298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043db32c-f4ea-45d1-ace4-fd54aa3cc0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-697a5872-0898-458e-ab1a-98322c271f87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:58.616516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:58.617464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:58.618509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:58.618509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:58.618509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:58.618509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:58.618509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:58.618509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:58.618509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:58.618509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:58.618509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:58.618509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:58.618509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:04:58.618509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:58.619398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043db32c-f4ea-45d1-ace4-fd54aa3cc0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.619398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043db32c-f4ea-45d1-ace4-fd54aa3cc0c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.619560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 246/495] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:04:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:58.619560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a5872-0898-458e-ab1a-98322c271f87 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:58.631516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043db32c-f4ea-45d1-ace4-fd54aa3cc0c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:58.632943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043db32c-f4ea-45d1-ace4-fd54aa3cc0c6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-043db32c-f4ea-45d1-ace4-fd54aa3cc0c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:04:58.647204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:58.648066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:04:58.648913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:04:58.648913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:58.648913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:58.648913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 250/496] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:58.649457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f5069-d4db-4be3-96b4-4c0f231b68e8 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.650046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f5069-d4db-4be3-96b4-4c0f231b68e8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.650415 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f5069-d4db-4be3-96b4-4c0f231b68e8 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.651554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 251/497] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:58.652567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02f9bb-a865-4d0f-98ef-1965e5534695 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.652860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02f9bb-a865-4d0f-98ef-1965e5534695 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.653122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02f9bb-a865-4d0f-98ef-1965e5534695 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.654400 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 252/498] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:04:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:04:58.654960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02f9bb-a865-4d0f-98ef-1965e5534695 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:58.656269 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:58.717739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:58.726450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a5872-0898-458e-ab1a-98322c271f87 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:58.757089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02f9bb-a865-4d0f-98ef-1965e5534695 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:58.759879 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d396d-a332-4159-8c62-c940187ed5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:58.760472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d396d-a332-4159-8c62-c940187ed5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.760778 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d396d-a332-4159-8c62-c940187ed5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.761046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d396d-a332-4159-8c62-c940187ed5c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.791668 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a77bf-fa81-4ccc-8023-ea642acc7bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:58.792287 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a77bf-fa81-4ccc-8023-ea642acc7bdd None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.792554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a77bf-fa81-4ccc-8023-ea642acc7bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.792819 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a77bf-fa81-4ccc-8023-ea642acc7bdd None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.793470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a77bf-fa81-4ccc-8023-ea642acc7bdd None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:58.793767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a77bf-fa81-4ccc-8023-ea642acc7bdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:58.795130 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a77bf-fa81-4ccc-8023-ea642acc7bdd None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:58.803535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 253/499] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:04:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:58.804465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a77bf-fa81-4ccc-8023-ea642acc7bdd None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:58.830229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d396d-a332-4159-8c62-c940187ed5c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:58.831684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d396d-a332-4159-8c62-c940187ed5c0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:58.862403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:58.906056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a77bf-fa81-4ccc-8023-ea642acc7bdd None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:58.932000 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:58.936300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 247/500] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:04:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:04:58.937588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d597b2-8a90-4d9d-abb3-57993ca47693 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.937867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d597b2-8a90-4d9d-abb3-57993ca47693 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.938253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d597b2-8a90-4d9d-abb3-57993ca47693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.941178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868cb32-ecd1-4771-b8e9-608e05fc6807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:58.941776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868cb32-ecd1-4771-b8e9-608e05fc6807 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.942043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868cb32-ecd1-4771-b8e9-608e05fc6807 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.942303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868cb32-ecd1-4771-b8e9-608e05fc6807 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.942942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868cb32-ecd1-4771-b8e9-608e05fc6807 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:58.943235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868cb32-ecd1-4771-b8e9-608e05fc6807 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:58.944788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868cb32-ecd1-4771-b8e9-608e05fc6807 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:58.956159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-40d597b2-8a90-4d9d-abb3-57993ca47693 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:58.957349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-40d597b2-8a90-4d9d-abb3-57993ca47693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:58.957761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 254/501] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:58.958932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b27c6-cb66-44bb-92a6-21aa232d5158 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:58.959208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b27c6-cb66-44bb-92a6-21aa232d5158 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:58.959472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b27c6-cb66-44bb-92a6-21aa232d5158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:58.979614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ed2b27c6-cb66-44bb-92a6-21aa232d5158 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:04:58.980812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ed2b27c6-cb66-44bb-92a6-21aa232d5158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:04:59.030965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 248/502] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:59.031653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d597b2-8a90-4d9d-abb3-57993ca47693 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:59.052678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 255/503] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:04:59.053550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b27c6-cb66-44bb-92a6-21aa232d5158 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:59.099353 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:59.154118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d597b2-8a90-4d9d-abb3-57993ca47693 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:59.169390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b27c6-cb66-44bb-92a6-21aa232d5158 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:59.186315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:59.194799 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050f757-9489-486c-be54-2672f2b15df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:59.195947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050f757-9489-486c-be54-2672f2b15df4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:59.196485 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050f757-9489-486c-be54-2672f2b15df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:59.197045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050f757-9489-486c-be54-2672f2b15df4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:59.205231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf56dd-fb09-4ca8-9a6b-ac0022e840d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:59.205853 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf56dd-fb09-4ca8-9a6b-ac0022e840d0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:59.206128 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf56dd-fb09-4ca8-9a6b-ac0022e840d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:59.206393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf56dd-fb09-4ca8-9a6b-ac0022e840d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:59.209348 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050f757-9489-486c-be54-2672f2b15df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:59.210925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050f757-9489-486c-be54-2672f2b15df4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:59.217188 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf56dd-fb09-4ca8-9a6b-ac0022e840d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:59.218715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf56dd-fb09-4ca8-9a6b-ac0022e840d0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-e050f757-9489-486c-be54-2672f2b15df4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:04:59.227965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:59.229056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:04:59.230109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:04:59.230109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:59.230109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-5ccf56dd-fb09-4ca8-9a6b-ac0022e840d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:04:59.236333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:59.237450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:04:59.238475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:04:59.238475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:59.238475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:59.238993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 249/504] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:59.240030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050f757-9489-486c-be54-2672f2b15df4 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:59.240920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 256/505] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:59.241774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf56dd-fb09-4ca8-9a6b-ac0022e840d0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:59.354264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf56dd-fb09-4ca8-9a6b-ac0022e840d0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:59.365131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050f757-9489-486c-be54-2672f2b15df4 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:59.383425 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:59.390018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96e59c-9281-4eb0-a5d0-3875ac73240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:59.390629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96e59c-9281-4eb0-a5d0-3875ac73240c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:59.390901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96e59c-9281-4eb0-a5d0-3875ac73240c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:59.391171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96e59c-9281-4eb0-a5d0-3875ac73240c None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:59.391818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96e59c-9281-4eb0-a5d0-3875ac73240c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:59.392117 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96e59c-9281-4eb0-a5d0-3875ac73240c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:59.393567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d96e59c-9281-4eb0-a5d0-3875ac73240c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:59.405093 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37968e95-047e-4dc5-852d-777b678992a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:59.405606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 257/506] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:59.405755 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37968e95-047e-4dc5-852d-777b678992a8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:59.405999 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37968e95-047e-4dc5-852d-777b678992a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:59.406270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37968e95-047e-4dc5-852d-777b678992a8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:59.406447 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96e59c-9281-4eb0-a5d0-3875ac73240c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:59.420258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37968e95-047e-4dc5-852d-777b678992a8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:59.422045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37968e95-047e-4dc5-852d-777b678992a8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:59.435201 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-37968e95-047e-4dc5-852d-777b678992a8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:04:59.435653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:59.436500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:59.437436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:59.437436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:59.437436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:04:59.437436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:04:59.437436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:04:59.437436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:04:59.437436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:04:59.437436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:04:59.437436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 15:04:59.437436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:59.437436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 250/507] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:04:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:59.437987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37968e95-047e-4dc5-852d-777b678992a8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:59.512343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96e59c-9281-4eb0-a5d0-3875ac73240c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:59.547821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37968e95-047e-4dc5-852d-777b678992a8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:59.550036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9f974-cfe9-4dad-a575-bd1c47690347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:59.550646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9f974-cfe9-4dad-a575-bd1c47690347 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:59.550921 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9f974-cfe9-4dad-a575-bd1c47690347 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:59.551182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9f974-cfe9-4dad-a575-bd1c47690347 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:59.551834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9f974-cfe9-4dad-a575-bd1c47690347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:59.552140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9f974-cfe9-4dad-a575-bd1c47690347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:59.553648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9f974-cfe9-4dad-a575-bd1c47690347 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:59.565938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 258/508] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:59.566863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9f974-cfe9-4dad-a575-bd1c47690347 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:59.582927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad05c3-ae3d-43c9-9a5b-0a73cd897ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:59.583562 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad05c3-ae3d-43c9-9a5b-0a73cd897ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:59.583833 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad05c3-ae3d-43c9-9a5b-0a73cd897ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:59.584099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad05c3-ae3d-43c9-9a5b-0a73cd897ac5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:59.595197 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad05c3-ae3d-43c9-9a5b-0a73cd897ac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-00ad05c3-ae3d-43c9-9a5b-0a73cd897ac5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:04:59.601360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:59.602228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:59.603183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:59.603183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:59.603183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:59.603183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:59.603183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:04:59.603183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:04:59.603183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:04:59.603183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:04:59.603183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:04:59.603183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:04:59.603183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:04:59.603183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:59.603183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 251/509] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:04:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:59.603995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad05c3-ae3d-43c9-9a5b-0a73cd897ac5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:59.638151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:59.662734 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:59.675621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9f974-cfe9-4dad-a575-bd1c47690347 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:59.706745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad05c3-ae3d-43c9-9a5b-0a73cd897ac5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:59.708102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac95a2-0093-4eb5-8fad-63bea59d3fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:59.708773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac95a2-0093-4eb5-8fad-63bea59d3fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:59.709051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac95a2-0093-4eb5-8fad-63bea59d3fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:59.709319 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac95a2-0093-4eb5-8fad-63bea59d3fa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:59.720220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac95a2-0093-4eb5-8fad-63bea59d3fa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:59.721733 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac95a2-0093-4eb5-8fad-63bea59d3fa2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-3eac95a2-0093-4eb5-8fad-63bea59d3fa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:04:59.735990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:59.736918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:04:59.737761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:04:59.737761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:04:59.737761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:04:59.737761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 259/510] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:59.738250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac95a2-0093-4eb5-8fad-63bea59d3fa2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:59.744717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3497a3-7a96-4bfa-8d00-ed8eb0b36387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:59.745392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3497a3-7a96-4bfa-8d00-ed8eb0b36387 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:59.745661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3497a3-7a96-4bfa-8d00-ed8eb0b36387 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:59.745928 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3497a3-7a96-4bfa-8d00-ed8eb0b36387 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:59.754502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3497a3-7a96-4bfa-8d00-ed8eb0b36387 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:59.756517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3497a3-7a96-4bfa-8d00-ed8eb0b36387 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-6f3497a3-7a96-4bfa-8d00-ed8eb0b36387 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:04:59.768063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:59.769159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:59.770038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:59.770038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:59.770038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:04:59.770038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:04:59.770038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:04:59.770038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:04:59.770038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:04:59.770038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:04:59.770038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:04:59.770038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:59.770038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 252/511] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Sep 23 15:04:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:59.770038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3497a3-7a96-4bfa-8d00-ed8eb0b36387 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:59.845291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac95a2-0093-4eb5-8fad-63bea59d3fa2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:59.861953 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:59.871130 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3497a3-7a96-4bfa-8d00-ed8eb0b36387 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:04:59.879729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f417b-3fc7-46d8-9901-33d1a0fc81f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:59.880329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f417b-3fc7-46d8-9901-33d1a0fc81f1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:59.880610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f417b-3fc7-46d8-9901-33d1a0fc81f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:59.880888 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f417b-3fc7-46d8-9901-33d1a0fc81f1 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:59.881618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f417b-3fc7-46d8-9901-33d1a0fc81f1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:04:59.881915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f417b-3fc7-46d8-9901-33d1a0fc81f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:59.883397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f417b-3fc7-46d8-9901-33d1a0fc81f1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:59.893126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:04:59.901989 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 260/512] 10.222.0.75 () {56 vars in 1137 bytes} [Fri Sep 23 15:04:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:04:59.902822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f417b-3fc7-46d8-9901-33d1a0fc81f1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:04:59.905465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeffde-cc41-442a-8a42-60250962db6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:04:59.906094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeffde-cc41-442a-8a42-60250962db6a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:04:59.906389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeffde-cc41-442a-8a42-60250962db6a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:04:59.906740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eeffde-cc41-442a-8a42-60250962db6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:04:59.913631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eeffde-cc41-442a-8a42-60250962db6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:04:59.915063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eeffde-cc41-442a-8a42-60250962db6a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-15eeffde-cc41-442a-8a42-60250962db6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:04:59.927620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:04:59.928484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:04:59.929354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:04:59.929354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:04:59.929354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:04:59.929354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:04:59.929354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:04:59.929354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:04:59.929354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:04:59.929354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:04:59.929354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 253/513] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:04:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:04:59.929823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeffde-cc41-442a-8a42-60250962db6a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:00.011121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f417b-3fc7-46d8-9901-33d1a0fc81f1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:00.031585 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eeffde-cc41-442a-8a42-60250962db6a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:00.045150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aacd2a-1ebb-4c99-9b13-c0a42ffbadfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:00.045768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aacd2a-1ebb-4c99-9b13-c0a42ffbadfe None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:00.046038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aacd2a-1ebb-4c99-9b13-c0a42ffbadfe None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:00.046299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aacd2a-1ebb-4c99-9b13-c0a42ffbadfe None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:00.046956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aacd2a-1ebb-4c99-9b13-c0a42ffbadfe None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:00.047252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aacd2a-1ebb-4c99-9b13-c0a42ffbadfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:00.048681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aacd2a-1ebb-4c99-9b13-c0a42ffbadfe None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:00.056536 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 261/514] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:04:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:00.057467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aacd2a-1ebb-4c99-9b13-c0a42ffbadfe None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:00.065151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca06809-483a-4791-8675-f28ba6ff5894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:00.065757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca06809-483a-4791-8675-f28ba6ff5894 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:00.066032 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca06809-483a-4791-8675-f28ba6ff5894 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:00.066296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca06809-483a-4791-8675-f28ba6ff5894 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:00.066982 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca06809-483a-4791-8675-f28ba6ff5894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:00.067279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca06809-483a-4791-8675-f28ba6ff5894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:00.068666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca06809-483a-4791-8675-f28ba6ff5894 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:00.080558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 254/515] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:00.081408 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca06809-483a-4791-8675-f28ba6ff5894 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:00.095467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:00.123677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:00.165446 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aacd2a-1ebb-4c99-9b13-c0a42ffbadfe None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:00.186487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca06809-483a-4791-8675-f28ba6ff5894 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:00.201590 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64dceff-2fc5-4ae9-acab-d144ef0b1302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:00.202220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dceff-2fc5-4ae9-acab-d144ef0b1302 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:00.202503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dceff-2fc5-4ae9-acab-d144ef0b1302 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:00.202774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dceff-2fc5-4ae9-acab-d144ef0b1302 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:00.203526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64dceff-2fc5-4ae9-acab-d144ef0b1302 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:00.203828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64dceff-2fc5-4ae9-acab-d144ef0b1302 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:00.205320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64dceff-2fc5-4ae9-acab-d144ef0b1302 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:00.220615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650e91e-d46a-422b-82ee-f18785fd4300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:00.221233 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650e91e-d46a-422b-82ee-f18785fd4300 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:00.221501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650e91e-d46a-422b-82ee-f18785fd4300 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:00.221763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650e91e-d46a-422b-82ee-f18785fd4300 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:00.223681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 262/516] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Sep 23 15:05:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:00.224339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64dceff-2fc5-4ae9-acab-d144ef0b1302 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:00.290813 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650e91e-d46a-422b-82ee-f18785fd4300 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:00.292257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650e91e-d46a-422b-82ee-f18785fd4300 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:00.306298 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:00.329808 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64dceff-2fc5-4ae9-acab-d144ef0b1302 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:00.348310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:00.367236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffec6d0-7bcc-427f-86ec-b37cb6eb745f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:00.367941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffec6d0-7bcc-427f-86ec-b37cb6eb745f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:00.368237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffec6d0-7bcc-427f-86ec-b37cb6eb745f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:00.368615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffec6d0-7bcc-427f-86ec-b37cb6eb745f None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:00.369356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffec6d0-7bcc-427f-86ec-b37cb6eb745f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:00.369655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffec6d0-7bcc-427f-86ec-b37cb6eb745f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:00.371122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffec6d0-7bcc-427f-86ec-b37cb6eb745f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:00.388920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 263/517] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:00.389762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffec6d0-7bcc-427f-86ec-b37cb6eb745f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:00.395085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 255/518] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:05:00.395921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650e91e-d46a-422b-82ee-f18785fd4300 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:00.492278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffec6d0-7bcc-427f-86ec-b37cb6eb745f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:00.494395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650e91e-d46a-422b-82ee-f18785fd4300 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:00.506872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:00.527825 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39cba4-8739-4884-8409-8efc77050cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:00.528194 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4045406-5656-4c75-8c7a-7f7613c1c3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:00.528423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39cba4-8739-4884-8409-8efc77050cc3 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:00.528733 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39cba4-8739-4884-8409-8efc77050cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:00.528873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4045406-5656-4c75-8c7a-7f7613c1c3cb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:00.528999 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39cba4-8739-4884-8409-8efc77050cc3 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:00.529129 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4045406-5656-4c75-8c7a-7f7613c1c3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:00.529391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4045406-5656-4c75-8c7a-7f7613c1c3cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:00.529641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39cba4-8739-4884-8409-8efc77050cc3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:00.531061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39cba4-8739-4884-8409-8efc77050cc3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:00.539651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4045406-5656-4c75-8c7a-7f7613c1c3cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:00.541613 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4045406-5656-4c75-8c7a-7f7613c1c3cb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:00.557250 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 256/519] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:05:00 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:05:00.558091 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39cba4-8739-4884-8409-8efc77050cc3 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d4045406-5656-4c75-8c7a-7f7613c1c3cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:00.559418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:00.560293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:00.561172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:00.561172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:00.561172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:00.561172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 264/520] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:00.561826 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4045406-5656-4c75-8c7a-7f7613c1c3cb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:00.611608 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:00.675991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39cba4-8739-4884-8409-8efc77050cc3 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:00.693961 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4045406-5656-4c75-8c7a-7f7613c1c3cb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:00.717402 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381676b-4d9f-4dc4-b6e3-1fb09321ff22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:00.718450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381676b-4d9f-4dc4-b6e3-1fb09321ff22 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:00.718964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381676b-4d9f-4dc4-b6e3-1fb09321ff22 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:00.719531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381676b-4d9f-4dc4-b6e3-1fb09321ff22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:00.730499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087afab0-5993-4f82-bc22-5fd988155051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:00.731503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087afab0-5993-4f82-bc22-5fd988155051 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:00.731768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087afab0-5993-4f82-bc22-5fd988155051 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:00.732041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087afab0-5993-4f82-bc22-5fd988155051 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:00.732801 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087afab0-5993-4f82-bc22-5fd988155051 None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=ac783fbe95ea4b74bd46af415c9734fc) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:00.733482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087afab0-5993-4f82-bc22-5fd988155051 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:00.734950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087afab0-5993-4f82-bc22-5fd988155051 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:00.740884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 265/521] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:00 2022] GET /identity/v3/role_assignments?user.id=ac783fbe95ea4b74bd46af415c9734fc&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:00.743699 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087afab0-5993-4f82-bc22-5fd988155051 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:00.776828 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:00.827572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b381676b-4d9f-4dc4-b6e3-1fb09321ff22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:00.829553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b381676b-4d9f-4dc4-b6e3-1fb09321ff22 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:00.877223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:00.891066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087afab0-5993-4f82-bc22-5fd988155051 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:00.937285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680aa2f4-4181-4eee-9bb7-d2e4fae41a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:00.938476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aa2f4-4181-4eee-9bb7-d2e4fae41a72 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:00.939367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aa2f4-4181-4eee-9bb7-d2e4fae41a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:00.939966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680aa2f4-4181-4eee-9bb7-d2e4fae41a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:00.953898 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680aa2f4-4181-4eee-9bb7-d2e4fae41a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:00.955360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680aa2f4-4181-4eee-9bb7-d2e4fae41a72 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:00.964516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 257/522] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:05:00.965476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381676b-4d9f-4dc4-b6e3-1fb09321ff22 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-680aa2f4-4181-4eee-9bb7-d2e4fae41a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:00.974561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:00.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:00.976424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:00.976424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:00.976424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:00.976424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 266/523] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:00.977162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680aa2f4-4181-4eee-9bb7-d2e4fae41a72 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:01.046305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:01.082751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381676b-4d9f-4dc4-b6e3-1fb09321ff22 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:01.111649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680aa2f4-4181-4eee-9bb7-d2e4fae41a72 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:01.120964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2feac-26b2-4b81-978b-24f194f9aba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:01.121780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2feac-26b2-4b81-978b-24f194f9aba9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:01.122149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2feac-26b2-4b81-978b-24f194f9aba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:01.122508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c2feac-26b2-4b81-978b-24f194f9aba9 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:01.123255 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2feac-26b2-4b81-978b-24f194f9aba9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:01.123645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2feac-26b2-4b81-978b-24f194f9aba9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:01.125130 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c2feac-26b2-4b81-978b-24f194f9aba9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:01.143681 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 258/524] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:01.144660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2feac-26b2-4b81-978b-24f194f9aba9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:01.151121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29b33b-a76c-41d4-a09a-59e19cd68737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:01.151761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29b33b-a76c-41d4-a09a-59e19cd68737 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:01.152034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29b33b-a76c-41d4-a09a-59e19cd68737 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:01.152302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29b33b-a76c-41d4-a09a-59e19cd68737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:01.164529 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29b33b-a76c-41d4-a09a-59e19cd68737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:01.166246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29b33b-a76c-41d4-a09a-59e19cd68737 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:01.173708 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-3b29b33b-a76c-41d4-a09a-59e19cd68737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:01.183601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:01.184605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:01.185537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:01.185537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:01.185537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:01.187641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 267/525] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:01.188680 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29b33b-a76c-41d4-a09a-59e19cd68737 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:01.267720 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c2feac-26b2-4b81-978b-24f194f9aba9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:01.293690 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29b33b-a76c-41d4-a09a-59e19cd68737 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:01.302726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:01.313873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70728a-2d21-42e7-81e5-fe5b55db0d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:01.314472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70728a-2d21-42e7-81e5-fe5b55db0d42 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:01.314736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70728a-2d21-42e7-81e5-fe5b55db0d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:01.314994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70728a-2d21-42e7-81e5-fe5b55db0d42 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:01.315641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70728a-2d21-42e7-81e5-fe5b55db0d42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:01.315998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70728a-2d21-42e7-81e5-fe5b55db0d42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:01.317989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70728a-2d21-42e7-81e5-fe5b55db0d42 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:01.331937 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c8968-677c-42a8-9b5e-991d6e79ef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:01.332864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 259/526] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:01.333268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c8968-677c-42a8-9b5e-991d6e79ef61 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:01.333810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70728a-2d21-42e7-81e5-fe5b55db0d42 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:01.333916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c8968-677c-42a8-9b5e-991d6e79ef61 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:01.334372 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c8968-677c-42a8-9b5e-991d6e79ef61 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:01.343634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c8968-677c-42a8-9b5e-991d6e79ef61 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:01.345449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c8968-677c-42a8-9b5e-991d6e79ef61 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-2c9c8968-677c-42a8-9b5e-991d6e79ef61 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:01.357505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:01.358735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:01.359602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:01.359602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:01.359602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:01.359602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:01.359602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:01.359602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:01.359602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:01.359602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:01.359602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 15:05:01.359602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:01.362296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 268/527] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:01.363340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c8968-677c-42a8-9b5e-991d6e79ef61 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:01.424982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:01.473989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70728a-2d21-42e7-81e5-fe5b55db0d42 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:01.483780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c8968-677c-42a8-9b5e-991d6e79ef61 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:01.510857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c82598-b097-4380-bba3-e41488f64c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:01.511554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c82598-b097-4380-bba3-e41488f64c86 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:01.511829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c82598-b097-4380-bba3-e41488f64c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:01.512095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c82598-b097-4380-bba3-e41488f64c86 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:01.512906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c82598-b097-4380-bba3-e41488f64c86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:01.513208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c82598-b097-4380-bba3-e41488f64c86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:01.514618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c82598-b097-4380-bba3-e41488f64c86 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:01.532561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 260/528] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:01.532561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c82598-b097-4380-bba3-e41488f64c86 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:01.541750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53d6da-c364-4b4e-9670-10da6d87b685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:01.542725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53d6da-c364-4b4e-9670-10da6d87b685 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:01.543218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53d6da-c364-4b4e-9670-10da6d87b685 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:01.543692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53d6da-c364-4b4e-9670-10da6d87b685 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:01.553482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53d6da-c364-4b4e-9670-10da6d87b685 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:01.556848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53d6da-c364-4b4e-9670-10da6d87b685 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-cc53d6da-c364-4b4e-9670-10da6d87b685 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:01.574687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:01.575634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:01.576481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:01.576481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:01.576481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:01.576481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:01.576481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:01.576481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:01.576481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:01.576481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:01.578094 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 269/529] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:01.579667 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53d6da-c364-4b4e-9670-10da6d87b685 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:01.595687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:01.659993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c82598-b097-4380-bba3-e41488f64c86 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:01.704873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53d6da-c364-4b4e-9670-10da6d87b685 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:01.706729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972abe61-bc64-4767-a228-bf929947d9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:01.707365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972abe61-bc64-4767-a228-bf929947d9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:01.707640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972abe61-bc64-4767-a228-bf929947d9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:01.707906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972abe61-bc64-4767-a228-bf929947d9d5 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:01.709322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972abe61-bc64-4767-a228-bf929947d9d5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:01.709322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972abe61-bc64-4767-a228-bf929947d9d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:01.710508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972abe61-bc64-4767-a228-bf929947d9d5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:01.726286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:01.747691 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cda06e-3d1b-42e5-92de-2923c43efb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:01.749261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cda06e-3d1b-42e5-92de-2923c43efb98 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:01.749756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cda06e-3d1b-42e5-92de-2923c43efb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:01.750460 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cda06e-3d1b-42e5-92de-2923c43efb98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:01.756700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 261/530] 10.222.0.75 () {56 vars in 1139 bytes} [Fri Sep 23 15:05:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:01.756700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972abe61-bc64-4767-a228-bf929947d9d5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:01.763927 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cda06e-3d1b-42e5-92de-2923c43efb98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:01.765673 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cda06e-3d1b-42e5-92de-2923c43efb98 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-58cda06e-3d1b-42e5-92de-2923c43efb98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:01.780640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:01.781636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:01.782497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:01.782497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:01.782497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:01.782497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:01.782497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:01.782497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:01.782497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:01.782497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:01.784729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 270/531] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:01.785692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cda06e-3d1b-42e5-92de-2923c43efb98 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:01.886153 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972abe61-bc64-4767-a228-bf929947d9d5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:01.893424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cda06e-3d1b-42e5-92de-2923c43efb98 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:01.894813 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:01.928199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5b312-0b11-4c7a-8616-dd0a630b9fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:01.928919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5b312-0b11-4c7a-8616-dd0a630b9fba None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:01.929275 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5b312-0b11-4c7a-8616-dd0a630b9fba None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:01.929605 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5b312-0b11-4c7a-8616-dd0a630b9fba None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:01.930364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5b312-0b11-4c7a-8616-dd0a630b9fba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:01.930768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5b312-0b11-4c7a-8616-dd0a630b9fba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:01.932283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5b312-0b11-4c7a-8616-dd0a630b9fba None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:01.933386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00265901-d2a8-406d-be4c-0086802ee87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:01.934313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00265901-d2a8-406d-be4c-0086802ee87c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:01.934783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00265901-d2a8-406d-be4c-0086802ee87c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:01.935271 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00265901-d2a8-406d-be4c-0086802ee87c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:01.956994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 262/532] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:01.957938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5b312-0b11-4c7a-8616-dd0a630b9fba None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:01.987737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:02.014088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00265901-d2a8-406d-be4c-0086802ee87c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:02.015843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00265901-d2a8-406d-be4c-0086802ee87c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:02.081629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5b312-0b11-4c7a-8616-dd0a630b9fba None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:02.125913 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83918a6d-3f74-47bc-b317-4a8571cf4e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:02.127150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83918a6d-3f74-47bc-b317-4a8571cf4e1f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:02.127534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83918a6d-3f74-47bc-b317-4a8571cf4e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:02.127927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83918a6d-3f74-47bc-b317-4a8571cf4e1f None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:02.129034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83918a6d-3f74-47bc-b317-4a8571cf4e1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:02.129439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83918a6d-3f74-47bc-b317-4a8571cf4e1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:02.131159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83918a6d-3f74-47bc-b317-4a8571cf4e1f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:02.150325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 263/533] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:02.150325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83918a6d-3f74-47bc-b317-4a8571cf4e1f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:02.157957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 271/534] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:05:02.158935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00265901-d2a8-406d-be4c-0086802ee87c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:02.188328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:02.242941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:02.267030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00265901-d2a8-406d-be4c-0086802ee87c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:02.271050 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83918a6d-3f74-47bc-b317-4a8571cf4e1f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:02.302686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70307af-1688-49a6-afa7-46ece9027f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:02.303425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70307af-1688-49a6-afa7-46ece9027f45 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:02.303797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70307af-1688-49a6-afa7-46ece9027f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:02.304168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70307af-1688-49a6-afa7-46ece9027f45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:02.305246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70307af-1688-49a6-afa7-46ece9027f45 None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=2af1ba390fff41a68ff1a0035901c318) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:02.305832 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70307af-1688-49a6-afa7-46ece9027f45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:02.307846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70307af-1688-49a6-afa7-46ece9027f45 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:02.308333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3b519-247b-41ba-8cff-de977939ba00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:02.309102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3b519-247b-41ba-8cff-de977939ba00 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:02.309523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3b519-247b-41ba-8cff-de977939ba00 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:02.309919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3b519-247b-41ba-8cff-de977939ba00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:02.310771 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3b519-247b-41ba-8cff-de977939ba00 None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=1cd51776db3d426ea1bf1133344fa187) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:02.311208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3b519-247b-41ba-8cff-de977939ba00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:02.312756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3b519-247b-41ba-8cff-de977939ba00 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:02.320616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 272/535] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:02 2022] GET /identity/v3/role_assignments?user.id=2af1ba390fff41a68ff1a0035901c318&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:02.320616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70307af-1688-49a6-afa7-46ece9027f45 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:02.330803 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 264/536] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:02 2022] GET /identity/v3/role_assignments?user.id=1cd51776db3d426ea1bf1133344fa187&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:02.487440 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70307af-1688-49a6-afa7-46ece9027f45 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:02.550531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059666e8-359d-43cc-b15a-2c6fe11f9c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:02.551243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059666e8-359d-43cc-b15a-2c6fe11f9c58 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:02.551519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059666e8-359d-43cc-b15a-2c6fe11f9c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:02.551789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059666e8-359d-43cc-b15a-2c6fe11f9c58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:02.563478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059666e8-359d-43cc-b15a-2c6fe11f9c58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:02.565024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059666e8-359d-43cc-b15a-2c6fe11f9c58 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:02.567251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-059666e8-359d-43cc-b15a-2c6fe11f9c58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:02.582714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:02.583679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:02.584607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:02.584607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:02.584607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:02.584607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 273/537] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:02.591680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3b519-247b-41ba-8cff-de977939ba00 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:02.692166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:02.705383 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3b519-247b-41ba-8cff-de977939ba00 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:02.742027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d184746-2c63-4f51-b171-3361afefe6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:02.742557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d184746-2c63-4f51-b171-3361afefe6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:02.742829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d184746-2c63-4f51-b171-3361afefe6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:02.743098 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d184746-2c63-4f51-b171-3361afefe6a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:02.743951 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d184746-2c63-4f51-b171-3361afefe6a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:02.744294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d184746-2c63-4f51-b171-3361afefe6a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:02.745959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d184746-2c63-4f51-b171-3361afefe6a6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:02.759598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 265/538] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:02.770042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059666e8-359d-43cc-b15a-2c6fe11f9c58 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:02.887181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059666e8-359d-43cc-b15a-2c6fe11f9c58 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:02.926323 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2e1ad-8a58-43a3-939a-098d7c005c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:02.927069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2e1ad-8a58-43a3-939a-098d7c005c8b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:02.927355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2e1ad-8a58-43a3-939a-098d7c005c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:02.927626 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2e1ad-8a58-43a3-939a-098d7c005c8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:02.936357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2e1ad-8a58-43a3-939a-098d7c005c8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:02.937928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2e1ad-8a58-43a3-939a-098d7c005c8b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-71b2e1ad-8a58-43a3-939a-098d7c005c8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:02.950851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:02.951756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:02.952567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:02.952567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:02.952567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:02.952567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:02.952567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:02.952567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:02.952567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:02.952567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:02.952567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 274/539] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:02.959499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d184746-2c63-4f51-b171-3361afefe6a6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:03.098075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d184746-2c63-4f51-b171-3361afefe6a6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:03.116357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85ce51-c255-4bd9-8cd4-c4bb3bad43b6 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:03.116735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85ce51-c255-4bd9-8cd4-c4bb3bad43b6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:03.116923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a85ce51-c255-4bd9-8cd4-c4bb3bad43b6 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:03.118655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 275/540] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:05:03.136379 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200702b-0632-40a0-bf37-ed22e4c65364 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:03.136714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200702b-0632-40a0-bf37-ed22e4c65364 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:03.137021 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200702b-0632-40a0-bf37-ed22e4c65364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:03.154100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab24068-0856-4ba9-ad34-cebf1e4e0909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:03.155126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab24068-0856-4ba9-ad34-cebf1e4e0909 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:03.155600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab24068-0856-4ba9-ad34-cebf1e4e0909 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:03.156079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab24068-0856-4ba9-ad34-cebf1e4e0909 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:03.157009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab24068-0856-4ba9-ad34-cebf1e4e0909 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:03.157515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab24068-0856-4ba9-ad34-cebf1e4e0909 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:03.158390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5200702b-0632-40a0-bf37-ed22e4c65364 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:03.159598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5200702b-0632-40a0-bf37-ed22e4c65364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:03.159941 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab24068-0856-4ba9-ad34-cebf1e4e0909 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:03.184315 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 266/541] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:03.190181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab24068-0856-4ba9-ad34-cebf1e4e0909 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:03.217097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:03.233526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 276/542] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:03.234081 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:03.250798 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151a92b-f677-40cc-b62d-410e190e5fba None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:03.250798 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151a92b-f677-40cc-b62d-410e190e5fba None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:03.250798 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151a92b-f677-40cc-b62d-410e190e5fba None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:03.252212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 277/543] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:03.256499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e1f42-bc2f-4728-9ac0-993d9bd799e4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:03.256816 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e1f42-bc2f-4728-9ac0-993d9bd799e4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:03.257072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e1f42-bc2f-4728-9ac0-993d9bd799e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:03.300619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-538e1f42-bc2f-4728-9ac0-993d9bd799e4 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:03.300619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-538e1f42-bc2f-4728-9ac0-993d9bd799e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:03.328389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab24068-0856-4ba9-ad34-cebf1e4e0909 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:03.370486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d813bc-b4dc-47fc-a338-9d1aac528cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:03.371024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d813bc-b4dc-47fc-a338-9d1aac528cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:03.371299 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d813bc-b4dc-47fc-a338-9d1aac528cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:03.371568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d813bc-b4dc-47fc-a338-9d1aac528cd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:03.372314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d813bc-b4dc-47fc-a338-9d1aac528cd1 None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=02afdedf8d394bbfa977ab7a22fafea7) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:03.372637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d813bc-b4dc-47fc-a338-9d1aac528cd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:03.374169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d813bc-b4dc-47fc-a338-9d1aac528cd1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:03.380422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 267/544] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:03 2022] GET /identity/v3/role_assignments?user.id=02afdedf8d394bbfa977ab7a22fafea7&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:03.426969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 278/545] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:03.579320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d813bc-b4dc-47fc-a338-9d1aac528cd1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:03.680838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:03.706641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d813bc-b4dc-47fc-a338-9d1aac528cd1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:03.745400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad9d24-f08e-4814-876c-0218a9949fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:03.746045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad9d24-f08e-4814-876c-0218a9949fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:03.746320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad9d24-f08e-4814-876c-0218a9949fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:03.746587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad9d24-f08e-4814-876c-0218a9949fe6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:03.757838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad9d24-f08e-4814-876c-0218a9949fe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-31ad9d24-f08e-4814-876c-0218a9949fe6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:03.764174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:03.765271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:03.766175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:03.766175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:03.766175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:03.766175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:03.766175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:03.766175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:03.766175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:03.766175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:03.766175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:03.766175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:03.766175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:03.766175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:03.766175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 268/546] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:03.773870 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad9d24-f08e-4814-876c-0218a9949fe6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:03.773999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e1f42-bc2f-4728-9ac0-993d9bd799e4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:03.890235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e1f42-bc2f-4728-9ac0-993d9bd799e4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:03.914998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:03.931703 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03fdb9-84ad-4cfa-b40c-270b529bd2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:03.932420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03fdb9-84ad-4cfa-b40c-270b529bd2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:03.932822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03fdb9-84ad-4cfa-b40c-270b529bd2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:03.933190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03fdb9-84ad-4cfa-b40c-270b529bd2b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:03.945446 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03fdb9-84ad-4cfa-b40c-270b529bd2b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-6f03fdb9-84ad-4cfa-b40c-270b529bd2b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:03.951202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:03.952281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:03.953387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:03.953387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:03.953387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:03.953387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:03.953387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:03.953387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:03.953387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:03.953387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:03.953387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:03.953387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:03.953387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:03.953387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:03.955526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 279/547] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:03.961633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03fdb9-84ad-4cfa-b40c-270b529bd2b7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:04.033016 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad9d24-f08e-4814-876c-0218a9949fe6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:04.100789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03fdb9-84ad-4cfa-b40c-270b529bd2b7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:04.106745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a9168-8334-4498-9161-9be2845c8df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:04.107361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a9168-8334-4498-9161-9be2845c8df9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:04.107632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a9168-8334-4498-9161-9be2845c8df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:04.107894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a9168-8334-4498-9161-9be2845c8df9 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:04.108909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a9168-8334-4498-9161-9be2845c8df9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:04.109276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a9168-8334-4498-9161-9be2845c8df9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:04.110649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a9168-8334-4498-9161-9be2845c8df9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:04.113560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:04.129160 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 269/548] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:04.138346 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a9168-8334-4498-9161-9be2845c8df9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:04.169390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858d97c-1c41-4a0d-8ef0-a265b80c8da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:04.170043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858d97c-1c41-4a0d-8ef0-a265b80c8da5 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:04.170272 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858d97c-1c41-4a0d-8ef0-a265b80c8da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:04.170533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858d97c-1c41-4a0d-8ef0-a265b80c8da5 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:04.171207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858d97c-1c41-4a0d-8ef0-a265b80c8da5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:04.171499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858d97c-1c41-4a0d-8ef0-a265b80c8da5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:04.178296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858d97c-1c41-4a0d-8ef0-a265b80c8da5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:04.193254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 280/549] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:04.198721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858d97c-1c41-4a0d-8ef0-a265b80c8da5 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:04.272951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:04.338943 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a9168-8334-4498-9161-9be2845c8df9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:04.395835 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858d97c-1c41-4a0d-8ef0-a265b80c8da5 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:04.402239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd449cb3-7449-4d45-97c9-92819d21bfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:04.407484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd449cb3-7449-4d45-97c9-92819d21bfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:04.408017 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd449cb3-7449-4d45-97c9-92819d21bfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:04.408487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd449cb3-7449-4d45-97c9-92819d21bfe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:04.423237 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd449cb3-7449-4d45-97c9-92819d21bfe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:04.426140 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd449cb3-7449-4d45-97c9-92819d21bfe2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-dd449cb3-7449-4d45-97c9-92819d21bfe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:04.450042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:04.451136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:04.451994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:04.451994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:04.451994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:04.460540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6632a-e6f6-454d-b434-51acbf1427e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:04.461490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6632a-e6f6-454d-b434-51acbf1427e7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:04.462019 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6632a-e6f6-454d-b434-51acbf1427e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:04.462487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6632a-e6f6-454d-b434-51acbf1427e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:04.463707 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 270/550] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:04.466074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd449cb3-7449-4d45-97c9-92819d21bfe2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:04.483732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6632a-e6f6-454d-b434-51acbf1427e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:04.485585 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6632a-e6f6-454d-b434-51acbf1427e7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-73f6632a-e6f6-454d-b434-51acbf1427e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:04.507342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:04.508321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:04.509425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:04.509425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:04.509425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:04.518218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 281/551] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:04.522250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6632a-e6f6-454d-b434-51acbf1427e7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:04.562033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:04.665061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd449cb3-7449-4d45-97c9-92819d21bfe2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:04.700554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:04.717621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6632a-e6f6-454d-b434-51acbf1427e7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:04.730564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6901fe-c9fd-44e4-83e8-71a58201dc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:04.731175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6901fe-c9fd-44e4-83e8-71a58201dc5e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:04.731443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6901fe-c9fd-44e4-83e8-71a58201dc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:04.731704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6901fe-c9fd-44e4-83e8-71a58201dc5e None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:04.732350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6901fe-c9fd-44e4-83e8-71a58201dc5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:04.732666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6901fe-c9fd-44e4-83e8-71a58201dc5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:04.734347 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6901fe-c9fd-44e4-83e8-71a58201dc5e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:04.758079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 271/552] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:04.764766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6901fe-c9fd-44e4-83e8-71a58201dc5e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:04.786267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e3045-71ad-4709-8239-e5befd6f5f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:04.786879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e3045-71ad-4709-8239-e5befd6f5f01 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:04.787144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e3045-71ad-4709-8239-e5befd6f5f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:04.787461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e3045-71ad-4709-8239-e5befd6f5f01 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:04.788116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e3045-71ad-4709-8239-e5befd6f5f01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:04.788409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e3045-71ad-4709-8239-e5befd6f5f01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:04.789995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e3045-71ad-4709-8239-e5befd6f5f01 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:04.816239 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 282/553] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:04.824618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e3045-71ad-4709-8239-e5befd6f5f01 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:04.965528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6901fe-c9fd-44e4-83e8-71a58201dc5e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:04.980954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:05.013623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e3045-71ad-4709-8239-e5befd6f5f01 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:05.032270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f69883-7668-4d5f-93d1-83b5d9f4f964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:05.032938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f69883-7668-4d5f-93d1-83b5d9f4f964 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:05.033217 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f69883-7668-4d5f-93d1-83b5d9f4f964 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:05.033491 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f69883-7668-4d5f-93d1-83b5d9f4f964 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:05.052735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f69883-7668-4d5f-93d1-83b5d9f4f964 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:05.052735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f69883-7668-4d5f-93d1-83b5d9f4f964 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-f7f69883-7668-4d5f-93d1-83b5d9f4f964 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:05.068399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:05.069615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:05.070665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:05.070665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:05.070665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:05.070665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:05.070665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:05.070665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:05.070665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:05.070665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:05.070665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 23 15:05:05.070665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:05.071764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 272/554] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:05:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:05.075575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52e7f0-d384-4af6-b2dd-22c60bd3fce5 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:05.075575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52e7f0-d384-4af6-b2dd-22c60bd3fce5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:05.075575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac52e7f0-d384-4af6-b2dd-22c60bd3fce5 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:05.080332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521290aa-aedd-430d-bebb-d3a847cafd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:05.080800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 273/555] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:05.081039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521290aa-aedd-430d-bebb-d3a847cafd4e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:05.081320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521290aa-aedd-430d-bebb-d3a847cafd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:05.081596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521290aa-aedd-430d-bebb-d3a847cafd4e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:05.082596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52e7f0-d384-4af6-b2dd-22c60bd3fce5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:05.090462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521290aa-aedd-430d-bebb-d3a847cafd4e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:05.091909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521290aa-aedd-430d-bebb-d3a847cafd4e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-521290aa-aedd-430d-bebb-d3a847cafd4e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:05.105432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:05.106451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:05.107827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:05.107827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:05.107827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:05.107827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:05.107827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:05.107827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:05.107827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:05.107827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:05.107827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 15:05:05.107827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:05.111643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 283/556] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:05:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:05.113021 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20901d03-9ae7-4038-a7c4-b9132ffd4b78 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:05.113288 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20901d03-9ae7-4038-a7c4-b9132ffd4b78 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:05.113620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20901d03-9ae7-4038-a7c4-b9132ffd4b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:05.139032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-20901d03-9ae7-4038-a7c4-b9132ffd4b78 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:05.140361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-20901d03-9ae7-4038-a7c4-b9132ffd4b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:05.143178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:05.210715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac52e7f0-d384-4af6-b2dd-22c60bd3fce5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:05.228848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 284/557] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:05.230443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20901d03-9ae7-4038-a7c4-b9132ffd4b78 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:05.268418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd9470-184a-4efa-abe0-3f6094ff766d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:05.269612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd9470-184a-4efa-abe0-3f6094ff766d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:05.270086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd9470-184a-4efa-abe0-3f6094ff766d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:05.270581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd9470-184a-4efa-abe0-3f6094ff766d None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:05.271694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd9470-184a-4efa-abe0-3f6094ff766d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:05.272348 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd9470-184a-4efa-abe0-3f6094ff766d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:05.274122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afd9470-184a-4efa-abe0-3f6094ff766d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:05.293616 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 274/558] 10.222.0.75 () {56 vars in 1137 bytes} [Fri Sep 23 15:05:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:05.299618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf34da5-1567-4129-8ed0-903020f19ae8 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:05.300142 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf34da5-1567-4129-8ed0-903020f19ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:05.300779 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf34da5-1567-4129-8ed0-903020f19ae8 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:05.302455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 275/559] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:05.303476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf34da5-1567-4129-8ed0-903020f19ae8 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:05.341331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:05.357825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20901d03-9ae7-4038-a7c4-b9132ffd4b78 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:05.410688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef70b3-0a81-436a-b91e-67c559fb8f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:05.411312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef70b3-0a81-436a-b91e-67c559fb8f27 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:05.411587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef70b3-0a81-436a-b91e-67c559fb8f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:05.411861 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef70b3-0a81-436a-b91e-67c559fb8f27 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:05.412900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef70b3-0a81-436a-b91e-67c559fb8f27 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:05.413204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef70b3-0a81-436a-b91e-67c559fb8f27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:05.414811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef70b3-0a81-436a-b91e-67c559fb8f27 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:05.427728 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:05.439165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf34da5-1567-4129-8ed0-903020f19ae8 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:05.441697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 285/560] 10.222.0.75 () {56 vars in 1135 bytes} [Fri Sep 23 15:05:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:05.443265 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84982b-629b-4f6a-aa30-bdafe530873a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:05.443795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84982b-629b-4f6a-aa30-bdafe530873a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:05.444260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84982b-629b-4f6a-aa30-bdafe530873a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:05.465641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-dd84982b-629b-4f6a-aa30-bdafe530873a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:05.467251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dd84982b-629b-4f6a-aa30-bdafe530873a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:05.478610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30749d00-0089-49cc-af8a-d717a6bd328c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:05.479294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30749d00-0089-49cc-af8a-d717a6bd328c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:05.479516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30749d00-0089-49cc-af8a-d717a6bd328c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:05.479916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30749d00-0089-49cc-af8a-d717a6bd328c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:05.493361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30749d00-0089-49cc-af8a-d717a6bd328c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:05.495071 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30749d00-0089-49cc-af8a-d717a6bd328c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-30749d00-0089-49cc-af8a-d717a6bd328c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:05.514337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:05.515344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:05.516247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:05.516247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:05.516247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:05.520486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 276/561] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:05.521414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30749d00-0089-49cc-af8a-d717a6bd328c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:05.550860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 286/562] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:05.551906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84982b-629b-4f6a-aa30-bdafe530873a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:05.618043 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:05.643980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30749d00-0089-49cc-af8a-d717a6bd328c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:05.668714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84982b-629b-4f6a-aa30-bdafe530873a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:05.682597 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bcdd6-290c-49b4-b0ba-48cb8ebeae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:05.683469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bcdd6-290c-49b4-b0ba-48cb8ebeae90 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:05.683743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bcdd6-290c-49b4-b0ba-48cb8ebeae90 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:05.684054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bcdd6-290c-49b4-b0ba-48cb8ebeae90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:05.697873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:05.708408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354cd13-8ca7-4dda-b377-fb429daf04bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:05.709352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354cd13-8ca7-4dda-b377-fb429daf04bc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:05.709874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354cd13-8ca7-4dda-b377-fb429daf04bc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:05.710335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354cd13-8ca7-4dda-b377-fb429daf04bc None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:05.711189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354cd13-8ca7-4dda-b377-fb429daf04bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:05.711713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354cd13-8ca7-4dda-b377-fb429daf04bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:05.713664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354cd13-8ca7-4dda-b377-fb429daf04bc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:05.726161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 287/563] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:05.727252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354cd13-8ca7-4dda-b377-fb429daf04bc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:05.762346 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bcdd6-290c-49b4-b0ba-48cb8ebeae90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:05.763727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bcdd6-290c-49b4-b0ba-48cb8ebeae90 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:05.836480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354cd13-8ca7-4dda-b377-fb429daf04bc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:05.858469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:05.873936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c23637-f147-45f3-b3ca-aeaa9243f2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:05.874568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c23637-f147-45f3-b3ca-aeaa9243f2ad None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:05.874846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c23637-f147-45f3-b3ca-aeaa9243f2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:05.875112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c23637-f147-45f3-b3ca-aeaa9243f2ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:05.881386 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 277/564] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:05:05.882366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bcdd6-290c-49b4-b0ba-48cb8ebeae90 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:05.892268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c23637-f147-45f3-b3ca-aeaa9243f2ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-a4c23637-f147-45f3-b3ca-aeaa9243f2ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:05.898787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:05.900223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 288/565] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:05.901121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c23637-f147-45f3-b3ca-aeaa9243f2ad None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:05.954760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:06.003294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bcdd6-290c-49b4-b0ba-48cb8ebeae90 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:06.012027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c23637-f147-45f3-b3ca-aeaa9243f2ad None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:06.036819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52eef78-1ba9-4af0-bd6a-5ee83e0e61e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:06.037346 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eef78-1ba9-4af0-bd6a-5ee83e0e61e3 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:06.037623 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eef78-1ba9-4af0-bd6a-5ee83e0e61e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:06.037888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eef78-1ba9-4af0-bd6a-5ee83e0e61e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:06.045169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52eef78-1ba9-4af0-bd6a-5ee83e0e61e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:06.045753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3ccd2-3d2e-4215-8036-fe2d0d2cd1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:06.046377 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3ccd2-3d2e-4215-8036-fe2d0d2cd1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:06.046628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52eef78-1ba9-4af0-bd6a-5ee83e0e61e3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:06.046719 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3ccd2-3d2e-4215-8036-fe2d0d2cd1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:06.046905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3ccd2-3d2e-4215-8036-fe2d0d2cd1b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:06.057228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3ccd2-3d2e-4215-8036-fe2d0d2cd1b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-a52eef78-1ba9-4af0-bd6a-5ee83e0e61e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:06.058710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:06.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:06.060399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:06.060399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:06.060399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:06.060399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:06.060399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:06.060399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:06.060399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:06.060399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:06.060843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 278/566] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:06.061666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52eef78-1ba9-4af0-bd6a-5ee83e0e61e3 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-c5b3ccd2-3d2e-4215-8036-fe2d0d2cd1b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:06.062385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:06.063282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:06.064091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:06.064091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:06.064091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:06.064091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:06.064091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:06.064091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:06.064091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:06.064091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:06.064091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:06.064091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:06.064091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:06.064091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:06.064091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 289/567] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:06.066798 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3ccd2-3d2e-4215-8036-fe2d0d2cd1b0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:06.102547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:06.165020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52eef78-1ba9-4af0-bd6a-5ee83e0e61e3 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:06.169421 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3ccd2-3d2e-4215-8036-fe2d0d2cd1b0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:06.176246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:06.199501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8b369-9204-446d-9446-59bcdbdfa682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:06.200115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8b369-9204-446d-9446-59bcdbdfa682 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:06.200387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8b369-9204-446d-9446-59bcdbdfa682 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:06.200674 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8b369-9204-446d-9446-59bcdbdfa682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:06.204860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943af39e-abd3-41de-b2c7-df76114f6477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:06.205464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943af39e-abd3-41de-b2c7-df76114f6477 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:06.205736 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943af39e-abd3-41de-b2c7-df76114f6477 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:06.206000 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943af39e-abd3-41de-b2c7-df76114f6477 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:06.206662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943af39e-abd3-41de-b2c7-df76114f6477 None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:06.206960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943af39e-abd3-41de-b2c7-df76114f6477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:06.208323 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943af39e-abd3-41de-b2c7-df76114f6477 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:06.210745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8b369-9204-446d-9446-59bcdbdfa682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:06.212341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8b369-9204-446d-9446-59bcdbdfa682 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:06.216286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 290/568] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:06.217495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24a9cd-0e25-4976-9049-0b8a9c24b0f6 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:06.217775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24a9cd-0e25-4976-9049-0b8a9c24b0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:06.218072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24a9cd-0e25-4976-9049-0b8a9c24b0f6 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:06.219445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 291/569] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:06.219933 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24a9cd-0e25-4976-9049-0b8a9c24b0f6 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-33b8b369-9204-446d-9446-59bcdbdfa682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:06.230902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:06.231923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:06.232853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:06.232853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:06.232853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:06.232853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 279/570] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:06.233400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8b369-9204-446d-9446-59bcdbdfa682 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:06.330531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:06.335754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24a9cd-0e25-4976-9049-0b8a9c24b0f6 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:06.340133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8b369-9204-446d-9446-59bcdbdfa682 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:06.370435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba2e8a-e62b-4e4a-b88c-038b46ee662d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:06.371036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba2e8a-e62b-4e4a-b88c-038b46ee662d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:06.371312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba2e8a-e62b-4e4a-b88c-038b46ee662d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:06.371577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba2e8a-e62b-4e4a-b88c-038b46ee662d None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:06.372260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba2e8a-e62b-4e4a-b88c-038b46ee662d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:06.372564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba2e8a-e62b-4e4a-b88c-038b46ee662d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:06.374015 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba2e8a-e62b-4e4a-b88c-038b46ee662d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:06.375478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce549d-eebd-4b33-a9c1-a1b95024fdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:06.376075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce549d-eebd-4b33-a9c1-a1b95024fdba None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:06.376333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce549d-eebd-4b33-a9c1-a1b95024fdba None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:06.376603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce549d-eebd-4b33-a9c1-a1b95024fdba None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:06.377282 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce549d-eebd-4b33-a9c1-a1b95024fdba None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:06.377577 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce549d-eebd-4b33-a9c1-a1b95024fdba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:06.378904 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce549d-eebd-4b33-a9c1-a1b95024fdba None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:06.386006 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 292/571] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:06.386787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba2e8a-e62b-4e4a-b88c-038b46ee662d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:06.391789 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 280/572] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:06.391789 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0542d6-98d9-4f75-99b3-308a115883f9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:06.391789 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0542d6-98d9-4f75-99b3-308a115883f9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:06.391789 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0542d6-98d9-4f75-99b3-308a115883f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:06.408314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ac0542d6-98d9-4f75-99b3-308a115883f9 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:06.409538 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ac0542d6-98d9-4f75-99b3-308a115883f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:06.421170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:06.477300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 281/573] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:06.478300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0542d6-98d9-4f75-99b3-308a115883f9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:06.509718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba2e8a-e62b-4e4a-b88c-038b46ee662d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:06.551274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ff8f8-3032-4f34-9c8c-734f9eef5643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:06.551912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ff8f8-3032-4f34-9c8c-734f9eef5643 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:06.552184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ff8f8-3032-4f34-9c8c-734f9eef5643 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:06.552620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ff8f8-3032-4f34-9c8c-734f9eef5643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:06.565126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:06.566022 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ff8f8-3032-4f34-9c8c-734f9eef5643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:06.567708 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ff8f8-3032-4f34-9c8c-734f9eef5643 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:06.583561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0542d6-98d9-4f75-99b3-308a115883f9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-0f8ff8f8-3032-4f34-9c8c-734f9eef5643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:06.585361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:06.586236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:06.587136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:06.587136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:06.587136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:06.587136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 293/574] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:06.587360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ff8f8-3032-4f34-9c8c-734f9eef5643 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:06.621133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e9551-1de9-4e3b-bc88-83a697767dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:06.622155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e9551-1de9-4e3b-bc88-83a697767dae None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:06.622619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e9551-1de9-4e3b-bc88-83a697767dae None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:06.623330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e9551-1de9-4e3b-bc88-83a697767dae None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:06.624867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e9551-1de9-4e3b-bc88-83a697767dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:06.625363 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e9551-1de9-4e3b-bc88-83a697767dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:06.627122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e9551-1de9-4e3b-bc88-83a697767dae None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:06.641371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 282/575] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:06.642633 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e9551-1de9-4e3b-bc88-83a697767dae None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:06.686072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:06.705478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ff8f8-3032-4f34-9c8c-734f9eef5643 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:06.740972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd05cd2-2591-4585-9e41-cb62433326ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:06.741596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd05cd2-2591-4585-9e41-cb62433326ec None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:06.741878 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd05cd2-2591-4585-9e41-cb62433326ec None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:06.742138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd05cd2-2591-4585-9e41-cb62433326ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:06.753277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd05cd2-2591-4585-9e41-cb62433326ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:06.754731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd05cd2-2591-4585-9e41-cb62433326ec None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:06.760734 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e9551-1de9-4e3b-bc88-83a697767dae None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-3dd05cd2-2591-4585-9e41-cb62433326ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:06.768923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:06.769867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:06.770769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:06.770769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:06.770769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:06.770769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 294/576] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:06.770999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd05cd2-2591-4585-9e41-cb62433326ec None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:06.799953 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a02e55-855a-4a67-b4e5-40408d85fbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:06.800682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02e55-855a-4a67-b4e5-40408d85fbea None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:06.800962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02e55-855a-4a67-b4e5-40408d85fbea None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:06.801232 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02e55-855a-4a67-b4e5-40408d85fbea None admin] PATH_INFO: `/v3/projects/27ac51280046479e82faaf3c0b550f72/users/ac783fbe95ea4b74bd46af415c9734fc/roles/351badda50da4e45ac1cb31482331c06` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:06.813447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:06.817034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a02e55-855a-4a67-b4e5-40408d85fbea None admin] RBAC: Authorizing `identity:create_grant(project_id=27ac51280046479e82faaf3c0b550f72, user_id=ac783fbe95ea4b74bd46af415c9734fc, role_id=351badda50da4e45ac1cb31482331c06)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:06.819158 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a02e55-855a-4a67-b4e5-40408d85fbea None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:06.832964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 283/577] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:05:06 2022] PUT /identity/v3/projects/27ac51280046479e82faaf3c0b550f72/users/ac783fbe95ea4b74bd46af415c9734fc/roles/351badda50da4e45ac1cb31482331c06 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:05:06.833976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a02e55-855a-4a67-b4e5-40408d85fbea None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:06.888251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd05cd2-2591-4585-9e41-cb62433326ec None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:06.921455 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:06.928002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507d7fe-eb6c-48ea-b4bb-7a5426c81d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:06.929063 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507d7fe-eb6c-48ea-b4bb-7a5426c81d3a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:06.929344 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507d7fe-eb6c-48ea-b4bb-7a5426c81d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:06.929616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507d7fe-eb6c-48ea-b4bb-7a5426c81d3a None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:06.930839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507d7fe-eb6c-48ea-b4bb-7a5426c81d3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:06.931162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507d7fe-eb6c-48ea-b4bb-7a5426c81d3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:06.932614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507d7fe-eb6c-48ea-b4bb-7a5426c81d3a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:06.945757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 295/578] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:06.946620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507d7fe-eb6c-48ea-b4bb-7a5426c81d3a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:06.959910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a02e55-855a-4a67-b4e5-40408d85fbea None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:07.002409 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6afdb-5959-4e2b-8c5f-1d93a6215c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:07.003010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6afdb-5959-4e2b-8c5f-1d93a6215c5f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:07.003281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6afdb-5959-4e2b-8c5f-1d93a6215c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:07.003577 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6afdb-5959-4e2b-8c5f-1d93a6215c5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:07.013144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6afdb-5959-4e2b-8c5f-1d93a6215c5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:07.013680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6afdb-5959-4e2b-8c5f-1d93a6215c5f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-7bf6afdb-5959-4e2b-8c5f-1d93a6215c5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:07.028753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:07.029687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:07.030575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:07.030575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:07.030575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:07.030575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:07.030575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:07.030575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:07.030575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:07.030575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:07.030575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 284/579] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:07.031089 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6afdb-5959-4e2b-8c5f-1d93a6215c5f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:07.067540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507d7fe-eb6c-48ea-b4bb-7a5426c81d3a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:07.090449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:07.105469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db04a1e-f988-4396-a9ef-e9a4de423e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:07.106107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db04a1e-f988-4396-a9ef-e9a4de423e32 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:07.106384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db04a1e-f988-4396-a9ef-e9a4de423e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:07.106652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db04a1e-f988-4396-a9ef-e9a4de423e32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:07.117753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db04a1e-f988-4396-a9ef-e9a4de423e32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-3db04a1e-f988-4396-a9ef-e9a4de423e32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:07.122577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:07.123473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 296/580] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:07.125013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db04a1e-f988-4396-a9ef-e9a4de423e32 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:07.153037 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6afdb-5959-4e2b-8c5f-1d93a6215c5f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:07.180270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:07.194474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082e72f-508f-4080-9a61-ffc606c4d9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:07.195090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082e72f-508f-4080-9a61-ffc606c4d9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:07.195487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082e72f-508f-4080-9a61-ffc606c4d9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:07.195760 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082e72f-508f-4080-9a61-ffc606c4d9a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:07.196509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082e72f-508f-4080-9a61-ffc606c4d9a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:07.196823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082e72f-508f-4080-9a61-ffc606c4d9a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:07.198203 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082e72f-508f-4080-9a61-ffc606c4d9a6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:07.213252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 285/581] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:07.214247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082e72f-508f-4080-9a61-ffc606c4d9a6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:07.237296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db04a1e-f988-4396-a9ef-e9a4de423e32 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:07.273901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e36576-1f6b-4530-8a3a-9facc56a6f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:07.274818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e36576-1f6b-4530-8a3a-9facc56a6f9c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:07.275371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e36576-1f6b-4530-8a3a-9facc56a6f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:07.275707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e36576-1f6b-4530-8a3a-9facc56a6f9c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:07.283075 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e36576-1f6b-4530-8a3a-9facc56a6f9c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:07.284527 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e36576-1f6b-4530-8a3a-9facc56a6f9c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-93e36576-1f6b-4530-8a3a-9facc56a6f9c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:07.296548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:07.297765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:07.298919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:07.298919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:07.298919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:07.298919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:07.298919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:07.298919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:07.298919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:07.298919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:07.298919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 15:05:07.298919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:07.298919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 297/582] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:05:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:07.301054 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e36576-1f6b-4530-8a3a-9facc56a6f9c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:07.324915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082e72f-508f-4080-9a61-ffc606c4d9a6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:07.339883 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:07.366684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816f30b-902e-4eb9-81f8-1f3d7caf6d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:07.367521 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816f30b-902e-4eb9-81f8-1f3d7caf6d92 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:07.367809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816f30b-902e-4eb9-81f8-1f3d7caf6d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:07.368422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816f30b-902e-4eb9-81f8-1f3d7caf6d92 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:07.369165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816f30b-902e-4eb9-81f8-1f3d7caf6d92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:07.369469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816f30b-902e-4eb9-81f8-1f3d7caf6d92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:07.370894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816f30b-902e-4eb9-81f8-1f3d7caf6d92 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:07.384051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 286/583] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:07.384936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816f30b-902e-4eb9-81f8-1f3d7caf6d92 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:07.423905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e36576-1f6b-4530-8a3a-9facc56a6f9c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:07.438298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:07.462084 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af840eac-35bd-40d4-9b61-7f2ceaa45144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:07.462756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af840eac-35bd-40d4-9b61-7f2ceaa45144 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:07.463066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af840eac-35bd-40d4-9b61-7f2ceaa45144 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:07.463334 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af840eac-35bd-40d4-9b61-7f2ceaa45144 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:07.464002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af840eac-35bd-40d4-9b61-7f2ceaa45144 None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:07.464297 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af840eac-35bd-40d4-9b61-7f2ceaa45144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:07.465707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af840eac-35bd-40d4-9b61-7f2ceaa45144 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:07.474197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 298/584] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:07.475003 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af840eac-35bd-40d4-9b61-7f2ceaa45144 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:07.505187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816f30b-902e-4eb9-81f8-1f3d7caf6d92 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:07.547017 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb02d8-1113-4bba-9317-195667ef71c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:07.548487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb02d8-1113-4bba-9317-195667ef71c0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:07.548768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb02d8-1113-4bba-9317-195667ef71c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:07.549042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb02d8-1113-4bba-9317-195667ef71c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:07.558745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb02d8-1113-4bba-9317-195667ef71c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:07.560373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb02d8-1113-4bba-9317-195667ef71c0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-90cb02d8-1113-4bba-9317-195667ef71c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:07.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:07.573609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:07.574737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:07.574737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:07.574737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:07.574737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:07.574737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:07.574737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:07.574737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:07.574737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:07.574737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:05:07.574737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:07.578033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 287/585] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Sep 23 15:05:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:07.578905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb02d8-1113-4bba-9317-195667ef71c0 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:07.597368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af840eac-35bd-40d4-9b61-7f2ceaa45144 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:07.613826 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:07.634562 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f29be0-d8ff-4b31-9bdd-3584268ceb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:07.634562 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f29be0-d8ff-4b31-9bdd-3584268ceb31 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:07.634830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f29be0-d8ff-4b31-9bdd-3584268ceb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:07.635042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f29be0-d8ff-4b31-9bdd-3584268ceb31 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:07.635781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f29be0-d8ff-4b31-9bdd-3584268ceb31 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:07.636069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f29be0-d8ff-4b31-9bdd-3584268ceb31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:07.637497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f29be0-d8ff-4b31-9bdd-3584268ceb31 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:07.659608 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 299/586] 10.222.0.75 () {56 vars in 1137 bytes} [Fri Sep 23 15:05:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:07.659608 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f29be0-d8ff-4b31-9bdd-3584268ceb31 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:07.692913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:07.701106 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb02d8-1113-4bba-9317-195667ef71c0 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:07.741164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b814e1e-6bd7-4c40-b993-f98d4f2e72ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:07.741883 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814e1e-6bd7-4c40-b993-f98d4f2e72ef None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:07.742211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814e1e-6bd7-4c40-b993-f98d4f2e72ef None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:07.742534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814e1e-6bd7-4c40-b993-f98d4f2e72ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:07.743387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b814e1e-6bd7-4c40-b993-f98d4f2e72ef None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=a8614f90cebb4a4eaf7ab5512927fb3c) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:07.743846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b814e1e-6bd7-4c40-b993-f98d4f2e72ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:07.745326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b814e1e-6bd7-4c40-b993-f98d4f2e72ef None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:07.750944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 288/587] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:07 2022] GET /identity/v3/role_assignments?user.id=a8614f90cebb4a4eaf7ab5512927fb3c&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:07.751601 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b814e1e-6bd7-4c40-b993-f98d4f2e72ef None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:07.788618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f29be0-d8ff-4b31-9bdd-3584268ceb31 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:07.831099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191233c-72c8-4ecc-ba38-dffd2d77c6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:07.831887 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191233c-72c8-4ecc-ba38-dffd2d77c6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:07.832066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191233c-72c8-4ecc-ba38-dffd2d77c6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:07.832321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191233c-72c8-4ecc-ba38-dffd2d77c6c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:07.844526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191233c-72c8-4ecc-ba38-dffd2d77c6c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:07.847194 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191233c-72c8-4ecc-ba38-dffd2d77c6c6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-4191233c-72c8-4ecc-ba38-dffd2d77c6c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:07.866814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:07.867837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:07.869238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:07.869238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:07.869238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:07.869238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 300/588] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:07.869478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191233c-72c8-4ecc-ba38-dffd2d77c6c6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:07.879455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:07.890202 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b814e1e-6bd7-4c40-b993-f98d4f2e72ef None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:07.944171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efcb84-1799-4b06-a0f1-137a1d9d6805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:07.944817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efcb84-1799-4b06-a0f1-137a1d9d6805 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:07.945094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efcb84-1799-4b06-a0f1-137a1d9d6805 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:07.945360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efcb84-1799-4b06-a0f1-137a1d9d6805 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:07.946103 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efcb84-1799-4b06-a0f1-137a1d9d6805 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:07.946486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efcb84-1799-4b06-a0f1-137a1d9d6805 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:07.948307 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efcb84-1799-4b06-a0f1-137a1d9d6805 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:07.965545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:07.970283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 289/589] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Sep 23 15:05:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:07.971380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efcb84-1799-4b06-a0f1-137a1d9d6805 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:07.987339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191233c-72c8-4ecc-ba38-dffd2d77c6c6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:08.027312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078544ca-2508-4c6a-9132-e63f42c4f260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:08.028490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078544ca-2508-4c6a-9132-e63f42c4f260 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:08.029414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078544ca-2508-4c6a-9132-e63f42c4f260 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:08.029834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078544ca-2508-4c6a-9132-e63f42c4f260 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:08.044458 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078544ca-2508-4c6a-9132-e63f42c4f260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:08.046479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078544ca-2508-4c6a-9132-e63f42c4f260 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-078544ca-2508-4c6a-9132-e63f42c4f260 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:08.065737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:08.066750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:08.068235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:08.068235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:08.068235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:08.068235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 301/590] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:08.068235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078544ca-2508-4c6a-9132-e63f42c4f260 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:08.089966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efcb84-1799-4b06-a0f1-137a1d9d6805 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:08.128052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2946a9-6756-4add-baac-4dd192fdc9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:08.128995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2946a9-6756-4add-baac-4dd192fdc9f8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:08.129446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2946a9-6756-4add-baac-4dd192fdc9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:08.129885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2946a9-6756-4add-baac-4dd192fdc9f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:08.130790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2946a9-6756-4add-baac-4dd192fdc9f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:08.131228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2946a9-6756-4add-baac-4dd192fdc9f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:08.132747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2946a9-6756-4add-baac-4dd192fdc9f8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:08.147328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 290/591] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:08.148338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2946a9-6756-4add-baac-4dd192fdc9f8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:08.169162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:08.195356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078544ca-2508-4c6a-9132-e63f42c4f260 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:08.234789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91517d-409d-4c0e-be6f-cc0dbedcc973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:08.235585 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91517d-409d-4c0e-be6f-cc0dbedcc973 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:08.235947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91517d-409d-4c0e-be6f-cc0dbedcc973 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:08.236307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91517d-409d-4c0e-be6f-cc0dbedcc973 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:08.239109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:08.250731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91517d-409d-4c0e-be6f-cc0dbedcc973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:08.252344 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa91517d-409d-4c0e-be6f-cc0dbedcc973 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-aa91517d-409d-4c0e-be6f-cc0dbedcc973 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:08.268114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:08.269343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:08.270381 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:08.270381 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:08.270381 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:08.272426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2946a9-6756-4add-baac-4dd192fdc9f8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:08.276414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 302/592] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:08.276684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91517d-409d-4c0e-be6f-cc0dbedcc973 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:08.310800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7c3c0-9e81-4a57-8181-a34f4b2cfcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:08.311803 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7c3c0-9e81-4a57-8181-a34f4b2cfcb9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:08.312278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7c3c0-9e81-4a57-8181-a34f4b2cfcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:08.312808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7c3c0-9e81-4a57-8181-a34f4b2cfcb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:08.313750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7c3c0-9e81-4a57-8181-a34f4b2cfcb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:08.314365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7c3c0-9e81-4a57-8181-a34f4b2cfcb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:08.315933 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7c3c0-9e81-4a57-8181-a34f4b2cfcb9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:08.329154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 291/593] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:08.330448 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7c3c0-9e81-4a57-8181-a34f4b2cfcb9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:08.399742 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91517d-409d-4c0e-be6f-cc0dbedcc973 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:08.416811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:08.446521 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7c3c0-9e81-4a57-8181-a34f4b2cfcb9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:08.446783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145e30d-0723-46b6-90a8-6bf91fed74a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:08.447499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145e30d-0723-46b6-90a8-6bf91fed74a4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:08.447728 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145e30d-0723-46b6-90a8-6bf91fed74a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:08.447990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145e30d-0723-46b6-90a8-6bf91fed74a4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:08.456016 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145e30d-0723-46b6-90a8-6bf91fed74a4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:08.457764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145e30d-0723-46b6-90a8-6bf91fed74a4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-2145e30d-0723-46b6-90a8-6bf91fed74a4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:08.469115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:08.470100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:08.471587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:08.471587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:08.471587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:08.471587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:08.471587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:08.471587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:08.471587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:08.471587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:08.471587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 15:05:08.471587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:08.471587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 303/594] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:08.471587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145e30d-0723-46b6-90a8-6bf91fed74a4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:08.497482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ca453-b2b4-4204-9f4a-99de038148ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:08.498143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ca453-b2b4-4204-9f4a-99de038148ef None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:08.498434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ca453-b2b4-4204-9f4a-99de038148ef None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:08.498713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ca453-b2b4-4204-9f4a-99de038148ef None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:08.499363 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ca453-b2b4-4204-9f4a-99de038148ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:08.499658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ca453-b2b4-4204-9f4a-99de038148ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:08.501255 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ca453-b2b4-4204-9f4a-99de038148ef None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:08.520437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 292/595] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:08.521419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ca453-b2b4-4204-9f4a-99de038148ef None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:08.532510 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:08.595986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145e30d-0723-46b6-90a8-6bf91fed74a4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:08.630824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980ecaa-36ee-40f3-ab66-93c62c0d5dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:08.631532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980ecaa-36ee-40f3-ab66-93c62c0d5dfb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:08.631802 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980ecaa-36ee-40f3-ab66-93c62c0d5dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:08.632106 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980ecaa-36ee-40f3-ab66-93c62c0d5dfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:08.639522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980ecaa-36ee-40f3-ab66-93c62c0d5dfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:08.640996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980ecaa-36ee-40f3-ab66-93c62c0d5dfb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-c980ecaa-36ee-40f3-ab66-93c62c0d5dfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:08.653030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:08.653951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:08.655457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:08.655457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:08.655457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:08.655457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:08.655457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:08.655457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:08.655457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:08.655457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:08.655457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 304/596] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:08.655457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980ecaa-36ee-40f3-ab66-93c62c0d5dfb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:08.656569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ca453-b2b4-4204-9f4a-99de038148ef None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:08.696916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175096d-c003-4ace-a079-7cde02908988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:08.697981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175096d-c003-4ace-a079-7cde02908988 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:08.698472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175096d-c003-4ace-a079-7cde02908988 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:08.699007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175096d-c003-4ace-a079-7cde02908988 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:08.699931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175096d-c003-4ace-a079-7cde02908988 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:08.700493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175096d-c003-4ace-a079-7cde02908988 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:08.702119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175096d-c003-4ace-a079-7cde02908988 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:08.705030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:08.725791 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 293/597] 10.222.0.75 () {56 vars in 1139 bytes} [Fri Sep 23 15:05:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:08.727252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175096d-c003-4ace-a079-7cde02908988 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:08.763155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980ecaa-36ee-40f3-ab66-93c62c0d5dfb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:08.772801 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:08.810555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2ac99-596b-47b4-9445-9f81d85db180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:08.811570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2ac99-596b-47b4-9445-9f81d85db180 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:08.812107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2ac99-596b-47b4-9445-9f81d85db180 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:08.812937 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2ac99-596b-47b4-9445-9f81d85db180 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:08.823249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2ac99-596b-47b4-9445-9f81d85db180 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:08.824750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2ac99-596b-47b4-9445-9f81d85db180 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-19d2ac99-596b-47b4-9445-9f81d85db180 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:08.836399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:08.837432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:08.838677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:08.838677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:08.838677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:08.838677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:08.838677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:08.838677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:08.838677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:08.838677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:08.838677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 305/598] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:08.838677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2ac99-596b-47b4-9445-9f81d85db180 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:08.862092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175096d-c003-4ace-a079-7cde02908988 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:08.909489 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873a919-a77f-4ade-b764-6a2fdef3bdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:08.910494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873a919-a77f-4ade-b764-6a2fdef3bdc8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:08.911021 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873a919-a77f-4ade-b764-6a2fdef3bdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:08.911515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873a919-a77f-4ade-b764-6a2fdef3bdc8 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:08.912796 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873a919-a77f-4ade-b764-6a2fdef3bdc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:08.913357 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873a919-a77f-4ade-b764-6a2fdef3bdc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:08.916242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873a919-a77f-4ade-b764-6a2fdef3bdc8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:08.930494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 294/599] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:08.932388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873a919-a77f-4ade-b764-6a2fdef3bdc8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:08.949886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2ac99-596b-47b4-9445-9f81d85db180 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:08.985163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effd133-d438-47e2-8763-c7a30812b578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:08.985942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effd133-d438-47e2-8763-c7a30812b578 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:08.986458 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effd133-d438-47e2-8763-c7a30812b578 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:08.986670 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effd133-d438-47e2-8763-c7a30812b578 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:08.997602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:08.999123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2effd133-d438-47e2-8763-c7a30812b578 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:09.000852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2effd133-d438-47e2-8763-c7a30812b578 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-2effd133-d438-47e2-8763-c7a30812b578 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:09.016915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:09.017910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:09.018803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:09.018803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:09.018803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:09.018803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 306/600] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:09.019022 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effd133-d438-47e2-8763-c7a30812b578 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:09.041159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:09.053378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873a919-a77f-4ade-b764-6a2fdef3bdc8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:09.089659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c39eb-d0c2-45c7-9d66-c2930c3c75b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:09.090268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c39eb-d0c2-45c7-9d66-c2930c3c75b0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:09.090536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c39eb-d0c2-45c7-9d66-c2930c3c75b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:09.090798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c39eb-d0c2-45c7-9d66-c2930c3c75b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:09.091529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c39eb-d0c2-45c7-9d66-c2930c3c75b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:09.091829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c39eb-d0c2-45c7-9d66-c2930c3c75b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:09.093293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c39eb-d0c2-45c7-9d66-c2930c3c75b0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:09.105681 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 295/601] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:09.106895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb8c55-a482-41ed-8144-84e960e49268 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:09.107166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb8c55-a482-41ed-8144-84e960e49268 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:09.107428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb8c55-a482-41ed-8144-84e960e49268 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:09.112743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 296/602] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:09.112743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb8c55-a482-41ed-8144-84e960e49268 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:09.134183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effd133-d438-47e2-8763-c7a30812b578 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:09.172948 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9ccdd-439e-4aba-9c4b-8e7d29de3143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:09.173540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9ccdd-439e-4aba-9c4b-8e7d29de3143 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:09.173813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9ccdd-439e-4aba-9c4b-8e7d29de3143 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:09.174087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9ccdd-439e-4aba-9c4b-8e7d29de3143 None admin] PATH_INFO: `/v3/projects/27ac51280046479e82faaf3c0b550f72/users/1cd51776db3d426ea1bf1133344fa187/roles/351badda50da4e45ac1cb31482331c06` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:09.189382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9ccdd-439e-4aba-9c4b-8e7d29de3143 None admin] RBAC: Authorizing `identity:create_grant(project_id=27ac51280046479e82faaf3c0b550f72, user_id=1cd51776db3d426ea1bf1133344fa187, role_id=351badda50da4e45ac1cb31482331c06)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:09.190820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9ccdd-439e-4aba-9c4b-8e7d29de3143 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:09.212626 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 307/603] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:05:09 2022] PUT /identity/v3/projects/27ac51280046479e82faaf3c0b550f72/users/1cd51776db3d426ea1bf1133344fa187/roles/351badda50da4e45ac1cb31482331c06 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:05:09.212626 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9ccdd-439e-4aba-9c4b-8e7d29de3143 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:09.237852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb8c55-a482-41ed-8144-84e960e49268 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:09.256385 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:09.283673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578ef2e-8f3a-4333-b8d3-9a8dde3b9400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:09.284380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578ef2e-8f3a-4333-b8d3-9a8dde3b9400 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:09.284752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578ef2e-8f3a-4333-b8d3-9a8dde3b9400 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:09.285334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578ef2e-8f3a-4333-b8d3-9a8dde3b9400 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:09.286003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578ef2e-8f3a-4333-b8d3-9a8dde3b9400 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:09.286302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578ef2e-8f3a-4333-b8d3-9a8dde3b9400 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:09.287749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578ef2e-8f3a-4333-b8d3-9a8dde3b9400 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:09.307100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 297/604] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:09.307100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9b9e3-ec86-48b2-98ba-7ebd32115dcb None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:09.307100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9b9e3-ec86-48b2-98ba-7ebd32115dcb None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:09.307100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9b9e3-ec86-48b2-98ba-7ebd32115dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:09.316999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:09.327201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-44b9b9e3-ec86-48b2-98ba-7ebd32115dcb None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:09.328628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-44b9b9e3-ec86-48b2-98ba-7ebd32115dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:09.345016 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9ccdd-439e-4aba-9c4b-8e7d29de3143 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:09.385499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7393a24-bcb1-4954-891a-92330c761a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:09.386539 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7393a24-bcb1-4954-891a-92330c761a47 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:09.387004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7393a24-bcb1-4954-891a-92330c761a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:09.387450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7393a24-bcb1-4954-891a-92330c761a47 None admin] PATH_INFO: `/v3/projects/27ac51280046479e82faaf3c0b550f72/users/2af1ba390fff41a68ff1a0035901c318/roles/351badda50da4e45ac1cb31482331c06` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:09.405122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7393a24-bcb1-4954-891a-92330c761a47 None admin] RBAC: Authorizing `identity:create_grant(project_id=27ac51280046479e82faaf3c0b550f72, user_id=2af1ba390fff41a68ff1a0035901c318, role_id=351badda50da4e45ac1cb31482331c06)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:09.407089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7393a24-bcb1-4954-891a-92330c761a47 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:09.412951 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 298/605] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:09.413947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9b9e3-ec86-48b2-98ba-7ebd32115dcb None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:09.435775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 308/606] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:05:09 2022] PUT /identity/v3/projects/27ac51280046479e82faaf3c0b550f72/users/2af1ba390fff41a68ff1a0035901c318/roles/351badda50da4e45ac1cb31482331c06 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:05:09.543804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9b9e3-ec86-48b2-98ba-7ebd32115dcb None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:09.548094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:09.583359 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d2bff-3e25-4096-acd0-b7c1a444f7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:09.583990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d2bff-3e25-4096-acd0-b7c1a444f7bd None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:09.584257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d2bff-3e25-4096-acd0-b7c1a444f7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:09.584658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d2bff-3e25-4096-acd0-b7c1a444f7bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:09.593390 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d2bff-3e25-4096-acd0-b7c1a444f7bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:09.594945 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d2bff-3e25-4096-acd0-b7c1a444f7bd None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-f28d2bff-3e25-4096-acd0-b7c1a444f7bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:09.607239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:09.608519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:09.609476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:09.609476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:09.609476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:09.609476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:09.609476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:09.609476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:09.609476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:09.609476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:09.609476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 299/607] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:09.621794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7393a24-bcb1-4954-891a-92330c761a47 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:09.752775 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d2bff-3e25-4096-acd0-b7c1a444f7bd None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:09.756764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7393a24-bcb1-4954-891a-92330c761a47 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:09.780457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:09.809551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccdffc-2798-4e04-a9a7-5415c79c4c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:09.810169 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccdffc-2798-4e04-a9a7-5415c79c4c90 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:09.810444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccdffc-2798-4e04-a9a7-5415c79c4c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:09.810707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ccdffc-2798-4e04-a9a7-5415c79c4c90 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:09.811403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ccdffc-2798-4e04-a9a7-5415c79c4c90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:09.811701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ccdffc-2798-4e04-a9a7-5415c79c4c90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:09.813146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ccdffc-2798-4e04-a9a7-5415c79c4c90 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:09.840803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 309/608] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:09.846643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccdffc-2798-4e04-a9a7-5415c79c4c90 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:09.957205 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d2bff-3e25-4096-acd0-b7c1a444f7bd None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:10.010415 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ccdffc-2798-4e04-a9a7-5415c79c4c90 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:10.049442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35700b5-704f-4a66-93a4-8e0977161d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:10.050743 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35700b5-704f-4a66-93a4-8e0977161d10 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:10.052047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e56f28-06b1-42a0-9c70-2b962f58637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:10.052562 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35700b5-704f-4a66-93a4-8e0977161d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:10.053383 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e56f28-06b1-42a0-9c70-2b962f58637e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:10.053830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35700b5-704f-4a66-93a4-8e0977161d10 None admin] PATH_INFO: `/v3/projects/27ac51280046479e82faaf3c0b550f72/users/02afdedf8d394bbfa977ab7a22fafea7/roles/351badda50da4e45ac1cb31482331c06` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:10.054805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e56f28-06b1-42a0-9c70-2b962f58637e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:10.055994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e56f28-06b1-42a0-9c70-2b962f58637e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:10.078120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35700b5-704f-4a66-93a4-8e0977161d10 None admin] RBAC: Authorizing `identity:create_grant(project_id=27ac51280046479e82faaf3c0b550f72, user_id=02afdedf8d394bbfa977ab7a22fafea7, role_id=351badda50da4e45ac1cb31482331c06)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:10.078617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e56f28-06b1-42a0-9c70-2b962f58637e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:10.079752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35700b5-704f-4a66-93a4-8e0977161d10 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-79e56f28-06b1-42a0-9c70-2b962f58637e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:10.084087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:10.085110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:10.085964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:10.085964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:10.085964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:10.085964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:10.085964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:10.085964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:10.085964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:10.085964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:10.085964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:10.085964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:10.085964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:10.085964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:10.088679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 300/609] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:10.090728 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4b581-bf47-42a1-a1c6-ba0b2011af0d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:10.093552 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4b581-bf47-42a1-a1c6-ba0b2011af0d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:10.093896 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4b581-bf47-42a1-a1c6-ba0b2011af0d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:10.100370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 301/610] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:10.100370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4b581-bf47-42a1-a1c6-ba0b2011af0d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:10.103264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 310/611] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:05:09 2022] PUT /identity/v3/projects/27ac51280046479e82faaf3c0b550f72/users/02afdedf8d394bbfa977ab7a22fafea7/roles/351badda50da4e45ac1cb31482331c06 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:05:10.104742 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a1d6-d2db-4531-9e53-d02059f899c4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:10.105279 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a1d6-d2db-4531-9e53-d02059f899c4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:10.105758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a1d6-d2db-4531-9e53-d02059f899c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:10.135336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-28d6a1d6-d2db-4531-9e53-d02059f899c4 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:10.138087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-28d6a1d6-d2db-4531-9e53-d02059f899c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:10.155979 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:10.192779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:10.266362 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 311/612] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:10.298480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4b581-bf47-42a1-a1c6-ba0b2011af0d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:10.339965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5df75b-2271-4862-9f16-a48ed514e40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:10.340572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5df75b-2271-4862-9f16-a48ed514e40e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:10.340868 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5df75b-2271-4862-9f16-a48ed514e40e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:10.341133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5df75b-2271-4862-9f16-a48ed514e40e None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:10.341805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5df75b-2271-4862-9f16-a48ed514e40e None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:10.342097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5df75b-2271-4862-9f16-a48ed514e40e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:10.343517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5df75b-2271-4862-9f16-a48ed514e40e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:10.353076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 302/613] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:10.361122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6a1d6-d2db-4531-9e53-d02059f899c4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:10.548173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6a1d6-d2db-4531-9e53-d02059f899c4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:10.606510 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836082d5-244f-422a-95c6-3483684db3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:10.607148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836082d5-244f-422a-95c6-3483684db3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:10.607419 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836082d5-244f-422a-95c6-3483684db3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:10.607684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836082d5-244f-422a-95c6-3483684db3b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:10.628252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836082d5-244f-422a-95c6-3483684db3b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:10.629757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836082d5-244f-422a-95c6-3483684db3b8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:10.632378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5df75b-2271-4862-9f16-a48ed514e40e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-836082d5-244f-422a-95c6-3483684db3b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:10.650268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:10.651276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:10.652154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:10.652154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:10.652154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:10.652154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 312/614] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:10.661457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836082d5-244f-422a-95c6-3483684db3b8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:10.686118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:10.737843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:10.755952 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5df75b-2271-4862-9f16-a48ed514e40e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:10.798383 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d261d8-4f5a-4e58-abc5-e3bfa0a06b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:10.798599 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d261d8-4f5a-4e58-abc5-e3bfa0a06b8e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:10.800621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d261d8-4f5a-4e58-abc5-e3bfa0a06b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:10.800621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d261d8-4f5a-4e58-abc5-e3bfa0a06b8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:10.823132 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d261d8-4f5a-4e58-abc5-e3bfa0a06b8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-a6d261d8-4f5a-4e58-abc5-e3bfa0a06b8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:10.833004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:10.834121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:10.835378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:10.835378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:10.835378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:10.835378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:10.835378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:10.835378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:10.835378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:10.835378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:10.835378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:10.835378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:10.835378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:10.835378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:10.835378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 303/615] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:10.842594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d261d8-4f5a-4e58-abc5-e3bfa0a06b8e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:10.846096 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836082d5-244f-422a-95c6-3483684db3b8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:10.914105 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fd5d6-2264-4a2a-8abe-78ae7b764b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:10.914712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fd5d6-2264-4a2a-8abe-78ae7b764b7b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:10.915045 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fd5d6-2264-4a2a-8abe-78ae7b764b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:10.915336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fd5d6-2264-4a2a-8abe-78ae7b764b7b None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:10.916081 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fd5d6-2264-4a2a-8abe-78ae7b764b7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:10.916377 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fd5d6-2264-4a2a-8abe-78ae7b764b7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:10.917777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fd5d6-2264-4a2a-8abe-78ae7b764b7b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:10.936849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 313/616] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:10.939468 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fd5d6-2264-4a2a-8abe-78ae7b764b7b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:11.036891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d261d8-4f5a-4e58-abc5-e3bfa0a06b8e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:11.073967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9e63c-79c7-4b7a-a455-affebef2d928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:11.075037 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9e63c-79c7-4b7a-a455-affebef2d928 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:11.075613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9e63c-79c7-4b7a-a455-affebef2d928 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:11.076086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9e63c-79c7-4b7a-a455-affebef2d928 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:11.077019 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9e63c-79c7-4b7a-a455-affebef2d928 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:11.077553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9e63c-79c7-4b7a-a455-affebef2d928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:11.079157 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9e63c-79c7-4b7a-a455-affebef2d928 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:11.088725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 304/617] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:11.097507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9e63c-79c7-4b7a-a455-affebef2d928 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:11.116943 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:11.119082 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:11.175812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fd5d6-2264-4a2a-8abe-78ae7b764b7b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:11.246312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a8442-0851-4686-b86f-5b7aef1a6581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:11.247734 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a8442-0851-4686-b86f-5b7aef1a6581 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:11.248408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a8442-0851-4686-b86f-5b7aef1a6581 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:11.248917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a8442-0851-4686-b86f-5b7aef1a6581 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:11.266632 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a8442-0851-4686-b86f-5b7aef1a6581 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:11.268380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a8442-0851-4686-b86f-5b7aef1a6581 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:11.271050 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9e63c-79c7-4b7a-a455-affebef2d928 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-cd9a8442-0851-4686-b86f-5b7aef1a6581 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:11.292226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:11.293285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:11.294337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:11.294337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:11.294337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:11.294337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:11.294337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:11.294337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:11.294337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:11.294337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:11.294337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 23 15:05:11.294337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:11.304969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 314/618] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:05:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:11.314273 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a8442-0851-4686-b86f-5b7aef1a6581 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:11.348303 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b6192-020f-4b99-95ea-865eae205167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:11.349732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b6192-020f-4b99-95ea-865eae205167 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:11.350201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b6192-020f-4b99-95ea-865eae205167 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:11.350679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b6192-020f-4b99-95ea-865eae205167 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:11.367610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b6192-020f-4b99-95ea-865eae205167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:11.370251 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b6192-020f-4b99-95ea-865eae205167 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-455b6192-020f-4b99-95ea-865eae205167 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:11.395686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:11.397167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:11.398070 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:11.398070 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:11.398070 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:11.404425 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 305/619] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:11.406013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b6192-020f-4b99-95ea-865eae205167 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:11.484998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:11.526152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a8442-0851-4686-b86f-5b7aef1a6581 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:11.581037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:11.586377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b6192-020f-4b99-95ea-865eae205167 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:11.588502 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a033f3-d108-4910-ba99-f38f85ce8f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:11.589146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a033f3-d108-4910-ba99-f38f85ce8f91 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:11.589420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a033f3-d108-4910-ba99-f38f85ce8f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:11.589686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a033f3-d108-4910-ba99-f38f85ce8f91 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:11.590423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a033f3-d108-4910-ba99-f38f85ce8f91 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:11.590723 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a033f3-d108-4910-ba99-f38f85ce8f91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:11.593025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a033f3-d108-4910-ba99-f38f85ce8f91 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:11.629975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 315/620] 10.222.0.75 () {56 vars in 1137 bytes} [Fri Sep 23 15:05:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:11.634073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a033f3-d108-4910-ba99-f38f85ce8f91 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:11.655989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad711aed-aad3-4801-9998-52df3dd0f2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:11.656681 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad711aed-aad3-4801-9998-52df3dd0f2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:11.656964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad711aed-aad3-4801-9998-52df3dd0f2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:11.657236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad711aed-aad3-4801-9998-52df3dd0f2d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:11.658218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad711aed-aad3-4801-9998-52df3dd0f2d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:11.658524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad711aed-aad3-4801-9998-52df3dd0f2d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:11.660159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad711aed-aad3-4801-9998-52df3dd0f2d9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:11.686046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 306/621] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:11.690127 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad711aed-aad3-4801-9998-52df3dd0f2d9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:11.830108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a033f3-d108-4910-ba99-f38f85ce8f91 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:11.867274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae0211-883c-4f02-a2d5-719f656c34ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:11.867889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0211-883c-4f02-a2d5-719f656c34ea None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:11.868390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0211-883c-4f02-a2d5-719f656c34ea None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:11.868529 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae0211-883c-4f02-a2d5-719f656c34ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:11.899901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad711aed-aad3-4801-9998-52df3dd0f2d9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:11.931347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:11.943587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae0211-883c-4f02-a2d5-719f656c34ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:11.944944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:11.945058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae0211-883c-4f02-a2d5-719f656c34ea None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:11.968118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919086e-b35b-40ed-9c9e-c35a5c017f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:11.968806 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919086e-b35b-40ed-9c9e-c35a5c017f25 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:11.969079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919086e-b35b-40ed-9c9e-c35a5c017f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:11.969347 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919086e-b35b-40ed-9c9e-c35a5c017f25 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:11.977687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919086e-b35b-40ed-9c9e-c35a5c017f25 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:11.979249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919086e-b35b-40ed-9c9e-c35a5c017f25 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-6919086e-b35b-40ed-9c9e-c35a5c017f25 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:11.992520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:11.994316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:11.995279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:11.995279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:11.995279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:11.995279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:11.995279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:11.995279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:11.995279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:11.995279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:11.995279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 15:05:11.995279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:11.995279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 307/622] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:05:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:11.996381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd5954-c86f-4d2b-8be2-d73c0959374a None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:11.996381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd5954-c86f-4d2b-8be2-d73c0959374a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:11.996715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd5954-c86f-4d2b-8be2-d73c0959374a None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.002026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 308/623] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:12.002937 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd5954-c86f-4d2b-8be2-d73c0959374a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:12.059616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 316/624] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 15:05:12.060969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd800863-6820-4400-b9c5-269731936269 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:12.061254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd800863-6820-4400-b9c5-269731936269 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:12.061518 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd800863-6820-4400-b9c5-269731936269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.080991 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-fd800863-6820-4400-b9c5-269731936269 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:12.082174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fd800863-6820-4400-b9c5-269731936269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:12.110424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd5954-c86f-4d2b-8be2-d73c0959374a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:12.145671 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86469f6-da26-40ff-b17a-ca74a7e27b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:12.146269 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86469f6-da26-40ff-b17a-ca74a7e27b43 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:12.146536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86469f6-da26-40ff-b17a-ca74a7e27b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:12.146797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86469f6-da26-40ff-b17a-ca74a7e27b43 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.147515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86469f6-da26-40ff-b17a-ca74a7e27b43 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:12.147810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86469f6-da26-40ff-b17a-ca74a7e27b43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:12.149267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86469f6-da26-40ff-b17a-ca74a7e27b43 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:12.153110 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 317/625] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:12.153699 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd800863-6820-4400-b9c5-269731936269 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:12.177188 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 309/626] 10.222.0.75 () {56 vars in 1135 bytes} [Fri Sep 23 15:05:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:12.179679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f18a8-c429-4c7f-8545-4347782ae99d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:12.179954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f18a8-c429-4c7f-8545-4347782ae99d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:12.180215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f18a8-c429-4c7f-8545-4347782ae99d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.182197 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 310/627] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:12.184934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f18a8-c429-4c7f-8545-4347782ae99d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:12.187183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:12.244961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:12.269817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd800863-6820-4400-b9c5-269731936269 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:12.307337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace34865-b464-45f9-80ed-2e5b8a82e94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:12.308142 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace34865-b464-45f9-80ed-2e5b8a82e94b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:12.308504 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace34865-b464-45f9-80ed-2e5b8a82e94b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:12.308903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace34865-b464-45f9-80ed-2e5b8a82e94b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.320395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace34865-b464-45f9-80ed-2e5b8a82e94b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:12.322144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace34865-b464-45f9-80ed-2e5b8a82e94b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:12.325572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f18a8-c429-4c7f-8545-4347782ae99d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-ace34865-b464-45f9-80ed-2e5b8a82e94b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:12.341247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:12.342285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:12.343200 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:12.343200 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:12.343200 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:12.343678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 318/628] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:12.347370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593107c-6d21-47be-aeb9-f8db6fc70e26 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:12.347370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593107c-6d21-47be-aeb9-f8db6fc70e26 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:12.347370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593107c-6d21-47be-aeb9-f8db6fc70e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.368723 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f593107c-6d21-47be-aeb9-f8db6fc70e26 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:12.370144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f593107c-6d21-47be-aeb9-f8db6fc70e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:12.372912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01086116-94b1-41fc-b6e9-54389db6958c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:12.374496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01086116-94b1-41fc-b6e9-54389db6958c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:12.374763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01086116-94b1-41fc-b6e9-54389db6958c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:12.375075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01086116-94b1-41fc-b6e9-54389db6958c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.389227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01086116-94b1-41fc-b6e9-54389db6958c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:12.390728 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01086116-94b1-41fc-b6e9-54389db6958c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-01086116-94b1-41fc-b6e9-54389db6958c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:12.410785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:12.412281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:12.413938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:12.413938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:12.413938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:12.416680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 311/629] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:12.417664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01086116-94b1-41fc-b6e9-54389db6958c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:12.444958 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 319/630] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:12.445855 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593107c-6d21-47be-aeb9-f8db6fc70e26 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:12.449024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:12.546595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:12.560985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01086116-94b1-41fc-b6e9-54389db6958c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:12.569733 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593107c-6d21-47be-aeb9-f8db6fc70e26 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:12.596432 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34e72e-309e-4efe-b015-5cd91ad75788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:12.597063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34e72e-309e-4efe-b015-5cd91ad75788 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:12.597326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34e72e-309e-4efe-b015-5cd91ad75788 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:12.597588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34e72e-309e-4efe-b015-5cd91ad75788 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.598224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34e72e-309e-4efe-b015-5cd91ad75788 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:12.598519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34e72e-309e-4efe-b015-5cd91ad75788 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:12.600176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34e72e-309e-4efe-b015-5cd91ad75788 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:12.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f97e7-3afa-4f19-a0c2-c10c14117730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:12.605432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f97e7-3afa-4f19-a0c2-c10c14117730 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:12.605702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f97e7-3afa-4f19-a0c2-c10c14117730 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:12.605967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f97e7-3afa-4f19-a0c2-c10c14117730 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.606625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f97e7-3afa-4f19-a0c2-c10c14117730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:12.607345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f97e7-3afa-4f19-a0c2-c10c14117730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:12.608654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f97e7-3afa-4f19-a0c2-c10c14117730 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:12.616761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 312/631] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:12.616761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34e72e-309e-4efe-b015-5cd91ad75788 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:12.623764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 320/632] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:12.625026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010fb87-949f-487d-b55d-35bbe2f7dd18 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:12.625302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010fb87-949f-487d-b55d-35bbe2f7dd18 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:12.625569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010fb87-949f-487d-b55d-35bbe2f7dd18 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.626928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 321/633] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:12.627460 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010fb87-949f-487d-b55d-35bbe2f7dd18 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:12.698493 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:12.727889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34e72e-309e-4efe-b015-5cd91ad75788 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:12.741408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010fb87-949f-487d-b55d-35bbe2f7dd18 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:12.763333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bafa6a-4c76-46f3-9b2c-df37f4d5dc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:12.763960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bafa6a-4c76-46f3-9b2c-df37f4d5dc6e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:12.764229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bafa6a-4c76-46f3-9b2c-df37f4d5dc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:12.764537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bafa6a-4c76-46f3-9b2c-df37f4d5dc6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.774750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bafa6a-4c76-46f3-9b2c-df37f4d5dc6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:12.774904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f60a9-54e6-46a3-b740-d35b61b0dfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:12.775454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f60a9-54e6-46a3-b740-d35b61b0dfe5 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:12.775723 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f60a9-54e6-46a3-b740-d35b61b0dfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:12.775987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f60a9-54e6-46a3-b740-d35b61b0dfe5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-73bafa6a-4c76-46f3-9b2c-df37f4d5dc6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:12.779444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:12.780456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 313/634] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:12.781321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bafa6a-4c76-46f3-9b2c-df37f4d5dc6e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:12.785752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f60a9-54e6-46a3-b740-d35b61b0dfe5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:12.787197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f60a9-54e6-46a3-b740-d35b61b0dfe5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:12.792640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7e4f60a9-54e6-46a3-b740-d35b61b0dfe5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:12.798885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:12.799788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:12.800681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:12.800681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:12.800681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:12.800681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:12.800681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:12.800681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:12.800681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:12.800681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:12.804703 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 322/635] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:12.806052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b936b-a69d-4656-9273-b33e3dd10491 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:12.806342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b936b-a69d-4656-9273-b33e3dd10491 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:12.806634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8b936b-a69d-4656-9273-b33e3dd10491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.824240 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4f8b936b-a69d-4656-9273-b33e3dd10491 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:12.825455 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4f8b936b-a69d-4656-9273-b33e3dd10491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:12.883718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bafa6a-4c76-46f3-9b2c-df37f4d5dc6e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:12.891467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 323/636] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:12.892305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b936b-a69d-4656-9273-b33e3dd10491 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:12.922280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d8748-1e9e-436e-a01e-bccee21835d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:12.922911 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d8748-1e9e-436e-a01e-bccee21835d1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:12.923181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d8748-1e9e-436e-a01e-bccee21835d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:12.923453 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d8748-1e9e-436e-a01e-bccee21835d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:12.930980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d8748-1e9e-436e-a01e-bccee21835d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:12.932566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d8748-1e9e-436e-a01e-bccee21835d1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-a93d8748-1e9e-436e-a01e-bccee21835d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:12.943797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:12.944926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:12.946020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:12.946020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:12.946020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:12.946020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:12.946020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:12.946020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:12.946020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:12.946020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:12.949512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 314/637] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:12.950593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d8748-1e9e-436e-a01e-bccee21835d1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:12.963598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:13.017801 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8b936b-a69d-4656-9273-b33e3dd10491 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:13.026375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:13.065615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c94df6-642c-4055-babe-049e458d04e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:13.066645 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c94df6-642c-4055-babe-049e458d04e8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:13.067324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c94df6-642c-4055-babe-049e458d04e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:13.067793 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c94df6-642c-4055-babe-049e458d04e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:13.069777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d8748-1e9e-436e-a01e-bccee21835d1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:13.081365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c94df6-642c-4055-babe-049e458d04e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-41c94df6-642c-4055-babe-049e458d04e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:13.088418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:13.089725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:13.090541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:13.090541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:13.090541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:13.090541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:13.090541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:13.090541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:13.090541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:13.090541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:13.090541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:13.090541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:13.090541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:13.090541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:13.092740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 324/638] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:13.093576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c94df6-642c-4055-babe-049e458d04e8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:13.115889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dea936-2e32-48fd-bca9-832e762832c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:13.117173 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dea936-2e32-48fd-bca9-832e762832c3 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:13.117683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dea936-2e32-48fd-bca9-832e762832c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:13.118285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dea936-2e32-48fd-bca9-832e762832c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:13.119195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dea936-2e32-48fd-bca9-832e762832c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:13.119762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dea936-2e32-48fd-bca9-832e762832c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:13.121686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dea936-2e32-48fd-bca9-832e762832c3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:13.136992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 315/639] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:13.138370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dea936-2e32-48fd-bca9-832e762832c3 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:13.216627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c94df6-642c-4055-babe-049e458d04e8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:13.229971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:13.255801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dea936-2e32-48fd-bca9-832e762832c3 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:13.257953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3658ce4-b089-409a-8c2a-c637c3cca1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:13.259553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3658ce4-b089-409a-8c2a-c637c3cca1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:13.259974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3658ce4-b089-409a-8c2a-c637c3cca1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:13.260454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3658ce4-b089-409a-8c2a-c637c3cca1b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:13.261320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3658ce4-b089-409a-8c2a-c637c3cca1b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:13.261782 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3658ce4-b089-409a-8c2a-c637c3cca1b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:13.263435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3658ce4-b089-409a-8c2a-c637c3cca1b2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:13.280933 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 325/640] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:13.282138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3658ce4-b089-409a-8c2a-c637c3cca1b2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:13.286451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:13.294587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93697aa-8959-4bfb-9064-d3e34e905e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:13.295207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93697aa-8959-4bfb-9064-d3e34e905e42 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:13.295483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93697aa-8959-4bfb-9064-d3e34e905e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:13.295748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93697aa-8959-4bfb-9064-d3e34e905e42 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:13.296449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93697aa-8959-4bfb-9064-d3e34e905e42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:13.296776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93697aa-8959-4bfb-9064-d3e34e905e42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:13.298215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93697aa-8959-4bfb-9064-d3e34e905e42 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:13.311202 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 316/641] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:13.312107 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93697aa-8959-4bfb-9064-d3e34e905e42 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:13.397671 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3658ce4-b089-409a-8c2a-c637c3cca1b2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:13.417376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93697aa-8959-4bfb-9064-d3e34e905e42 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:13.433242 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48bb170-9024-4538-9a5a-37deaee67ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:13.433867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48bb170-9024-4538-9a5a-37deaee67ccc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:13.434131 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48bb170-9024-4538-9a5a-37deaee67ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:13.434392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48bb170-9024-4538-9a5a-37deaee67ccc None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:13.435055 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48bb170-9024-4538-9a5a-37deaee67ccc None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:13.435350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48bb170-9024-4538-9a5a-37deaee67ccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:13.436737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48bb170-9024-4538-9a5a-37deaee67ccc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:13.444886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 326/642] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:13.445680 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48bb170-9024-4538-9a5a-37deaee67ccc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:13.460481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab59d13-9c44-432e-9349-813e173ee43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:13.461144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab59d13-9c44-432e-9349-813e173ee43a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:13.461411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab59d13-9c44-432e-9349-813e173ee43a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:13.461811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab59d13-9c44-432e-9349-813e173ee43a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:13.473193 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab59d13-9c44-432e-9349-813e173ee43a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:13.474606 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab59d13-9c44-432e-9349-813e173ee43a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:13.484528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-fab59d13-9c44-432e-9349-813e173ee43a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:13.489983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:13.491147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:13.492051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:13.492051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:13.492051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:13.492051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 317/643] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:13.492051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab59d13-9c44-432e-9349-813e173ee43a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:13.541579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:13.557741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48bb170-9024-4538-9a5a-37deaee67ccc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:13.590679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb6550-7964-447a-a7f2-be82ddbf559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:13.591306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb6550-7964-447a-a7f2-be82ddbf559b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:13.591574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb6550-7964-447a-a7f2-be82ddbf559b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:13.591836 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb6550-7964-447a-a7f2-be82ddbf559b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:13.597440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab59d13-9c44-432e-9349-813e173ee43a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:13.601442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb6550-7964-447a-a7f2-be82ddbf559b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7deb6550-7964-447a-a7f2-be82ddbf559b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:13.606570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:13.607688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 327/644] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:13.609044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb6550-7964-447a-a7f2-be82ddbf559b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:13.631879 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bcea0-3309-48fa-8cf2-45e6def0a40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:13.632602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bcea0-3309-48fa-8cf2-45e6def0a40a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:13.632881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bcea0-3309-48fa-8cf2-45e6def0a40a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:13.633151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bcea0-3309-48fa-8cf2-45e6def0a40a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:13.634423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bcea0-3309-48fa-8cf2-45e6def0a40a None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=ac783fbe95ea4b74bd46af415c9734fc) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:13.634732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bcea0-3309-48fa-8cf2-45e6def0a40a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:13.636148 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bcea0-3309-48fa-8cf2-45e6def0a40a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:13.642046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 318/645] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:13 2022] GET /identity/v3/role_assignments?user.id=ac783fbe95ea4b74bd46af415c9734fc&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:13.642646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bcea0-3309-48fa-8cf2-45e6def0a40a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:13.720838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:13.729035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb6550-7964-447a-a7f2-be82ddbf559b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:13.766935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a35cbe-7127-420e-b9e2-c428f1c7af95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:13.767617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a35cbe-7127-420e-b9e2-c428f1c7af95 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:13.767896 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a35cbe-7127-420e-b9e2-c428f1c7af95 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:13.768166 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a35cbe-7127-420e-b9e2-c428f1c7af95 None admin] PATH_INFO: `/v3/projects/27ac51280046479e82faaf3c0b550f72/users/a8614f90cebb4a4eaf7ab5512927fb3c/roles/351badda50da4e45ac1cb31482331c06` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:13.768747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bcea0-3309-48fa-8cf2-45e6def0a40a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:13.778052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:13.786688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a35cbe-7127-420e-b9e2-c428f1c7af95 None admin] RBAC: Authorizing `identity:create_grant(project_id=27ac51280046479e82faaf3c0b550f72, user_id=a8614f90cebb4a4eaf7ab5512927fb3c, role_id=351badda50da4e45ac1cb31482331c06)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:13.788166 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a35cbe-7127-420e-b9e2-c428f1c7af95 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:13.812624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 328/646] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:05:13 2022] PUT /identity/v3/projects/27ac51280046479e82faaf3c0b550f72/users/a8614f90cebb4a4eaf7ab5512927fb3c/roles/351badda50da4e45ac1cb31482331c06 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:05:13.812624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a35cbe-7127-420e-b9e2-c428f1c7af95 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:13.817977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b619c28-0ac9-4447-a6cd-f97a3f7424a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:13.818697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b619c28-0ac9-4447-a6cd-f97a3f7424a4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:13.818960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b619c28-0ac9-4447-a6cd-f97a3f7424a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:13.819224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b619c28-0ac9-4447-a6cd-f97a3f7424a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:13.832059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b619c28-0ac9-4447-a6cd-f97a3f7424a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:13.833701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b619c28-0ac9-4447-a6cd-f97a3f7424a4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-0b619c28-0ac9-4447-a6cd-f97a3f7424a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:13.862312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:13.863339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:13.864227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:13.864227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:13.864227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:13.864227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 319/647] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:13.864227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b619c28-0ac9-4447-a6cd-f97a3f7424a4 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:13.963940 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a35cbe-7127-420e-b9e2-c428f1c7af95 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:14.005063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b619c28-0ac9-4447-a6cd-f97a3f7424a4 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:14.007695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb4f83-da7e-46c7-9601-770c0c06c836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:14.010759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb4f83-da7e-46c7-9601-770c0c06c836 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:14.010759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb4f83-da7e-46c7-9601-770c0c06c836 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:14.010759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb4f83-da7e-46c7-9601-770c0c06c836 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:14.010759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb4f83-da7e-46c7-9601-770c0c06c836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:14.010759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb4f83-da7e-46c7-9601-770c0c06c836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:14.011648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb4f83-da7e-46c7-9601-770c0c06c836 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:14.031895 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 329/648] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:14.032950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb4f83-da7e-46c7-9601-770c0c06c836 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:14.037731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:14.052380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9a5a4-3c42-482c-b2c4-d97b1215f411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:14.054197 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9a5a4-3c42-482c-b2c4-d97b1215f411 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:14.054648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9a5a4-3c42-482c-b2c4-d97b1215f411 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:14.055162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9a5a4-3c42-482c-b2c4-d97b1215f411 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:14.056534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9a5a4-3c42-482c-b2c4-d97b1215f411 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:14.057191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9a5a4-3c42-482c-b2c4-d97b1215f411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:14.058895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9a5a4-3c42-482c-b2c4-d97b1215f411 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:14.068527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 320/649] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:14.069686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9a5a4-3c42-482c-b2c4-d97b1215f411 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:14.095043 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:14.161042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb4f83-da7e-46c7-9601-770c0c06c836 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:14.189117 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9a5a4-3c42-482c-b2c4-d97b1215f411 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:14.198886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750bb46-7625-49b3-bf6b-36e0424c0870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:14.199497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750bb46-7625-49b3-bf6b-36e0424c0870 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:14.199776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750bb46-7625-49b3-bf6b-36e0424c0870 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:14.200047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750bb46-7625-49b3-bf6b-36e0424c0870 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:14.200731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750bb46-7625-49b3-bf6b-36e0424c0870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:14.201035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750bb46-7625-49b3-bf6b-36e0424c0870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:14.202592 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d750bb46-7625-49b3-bf6b-36e0424c0870 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:14.220371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 330/650] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:14.220371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750bb46-7625-49b3-bf6b-36e0424c0870 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:14.236418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0d074-99cc-46fd-9493-6f03ef9e8e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:14.237325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0d074-99cc-46fd-9493-6f03ef9e8e7c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:14.237767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0d074-99cc-46fd-9493-6f03ef9e8e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:14.238176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0d074-99cc-46fd-9493-6f03ef9e8e7c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:14.249582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0d074-99cc-46fd-9493-6f03ef9e8e7c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:14.252054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0d074-99cc-46fd-9493-6f03ef9e8e7c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-63d0d074-99cc-46fd-9493-6f03ef9e8e7c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:14.266054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:14.267035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:14.267959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:14.267959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:14.267959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:14.267959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:14.267959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:14.267959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:14.267959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:14.267959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:14.267959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 15:05:14.267959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:14.267959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 321/651] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:05:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:14.267959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0d074-99cc-46fd-9493-6f03ef9e8e7c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:14.301009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:14.343470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750bb46-7625-49b3-bf6b-36e0424c0870 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:14.376413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:14.388275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef7133-3fe1-4446-b2ef-66a1888c5a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:14.389464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef7133-3fe1-4446-b2ef-66a1888c5a31 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:14.390201 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef7133-3fe1-4446-b2ef-66a1888c5a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:14.390898 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fef7133-3fe1-4446-b2ef-66a1888c5a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:14.399094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0d074-99cc-46fd-9493-6f03ef9e8e7c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:14.405049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef7133-3fe1-4446-b2ef-66a1888c5a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:14.406735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fef7133-3fe1-4446-b2ef-66a1888c5a31 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7fef7133-3fe1-4446-b2ef-66a1888c5a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:14.425874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:14.427017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:14.428078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:14.428078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:14.428078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:14.435584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 331/652] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:14.435584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef7133-3fe1-4446-b2ef-66a1888c5a31 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:14.447697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019bd01-79f6-4104-be4d-2b5b4bffb667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:14.448570 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019bd01-79f6-4104-be4d-2b5b4bffb667 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:14.449453 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019bd01-79f6-4104-be4d-2b5b4bffb667 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:14.449831 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019bd01-79f6-4104-be4d-2b5b4bffb667 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:14.459678 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019bd01-79f6-4104-be4d-2b5b4bffb667 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:14.461883 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019bd01-79f6-4104-be4d-2b5b4bffb667 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-4019bd01-79f6-4104-be4d-2b5b4bffb667 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:14.476834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:14.478664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:14.479805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:14.479805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:14.479805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:14.479805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:14.479805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:14.479805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:14.479805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:14.479805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:14.479805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:05:14.479805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:14.485825 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 322/653] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Sep 23 15:05:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:14.485825 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019bd01-79f6-4104-be4d-2b5b4bffb667 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:14.572876 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fef7133-3fe1-4446-b2ef-66a1888c5a31 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:14.591586 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:14.604020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019bd01-79f6-4104-be4d-2b5b4bffb667 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:14.613903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36e602-31c0-49b7-9f18-fcdc1f48001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:14.614432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36e602-31c0-49b7-9f18-fcdc1f48001f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:14.614827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36e602-31c0-49b7-9f18-fcdc1f48001f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:14.615197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36e602-31c0-49b7-9f18-fcdc1f48001f None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:14.616133 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36e602-31c0-49b7-9f18-fcdc1f48001f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:14.616620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36e602-31c0-49b7-9f18-fcdc1f48001f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:14.618193 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36e602-31c0-49b7-9f18-fcdc1f48001f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:14.639865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 332/654] 10.222.0.75 () {56 vars in 1137 bytes} [Fri Sep 23 15:05:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:14.640226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb6caa-bae0-4616-a81b-a5e425516681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:14.641004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36e602-31c0-49b7-9f18-fcdc1f48001f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:14.641261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb6caa-bae0-4616-a81b-a5e425516681 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:14.641669 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb6caa-bae0-4616-a81b-a5e425516681 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:14.642072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb6caa-bae0-4616-a81b-a5e425516681 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:14.642855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb6caa-bae0-4616-a81b-a5e425516681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:14.643292 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb6caa-bae0-4616-a81b-a5e425516681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:14.645161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb6caa-bae0-4616-a81b-a5e425516681 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:14.654622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:14.662900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 323/655] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:14.663795 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb6caa-bae0-4616-a81b-a5e425516681 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:14.759687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36e602-31c0-49b7-9f18-fcdc1f48001f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:14.767342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb6caa-bae0-4616-a81b-a5e425516681 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:14.798990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7a6c5-8ab4-4b5a-819a-6d7061d59939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:14.800126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7a6c5-8ab4-4b5a-819a-6d7061d59939 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:14.800666 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7a6c5-8ab4-4b5a-819a-6d7061d59939 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:14.801246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7a6c5-8ab4-4b5a-819a-6d7061d59939 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:14.801646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd05fd2-96cf-43c8-8dc1-1a71359a00ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:14.802244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd05fd2-96cf-43c8-8dc1-1a71359a00ca None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:14.802514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd05fd2-96cf-43c8-8dc1-1a71359a00ca None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:14.802778 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd05fd2-96cf-43c8-8dc1-1a71359a00ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:14.803062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7a6c5-8ab4-4b5a-819a-6d7061d59939 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:14.803565 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7a6c5-8ab4-4b5a-819a-6d7061d59939 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:14.805347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7a6c5-8ab4-4b5a-819a-6d7061d59939 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:14.829295 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 333/656] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Sep 23 15:05:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:14.830565 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7a6c5-8ab4-4b5a-819a-6d7061d59939 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:14.865903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:14.869693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:14.873022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd05fd2-96cf-43c8-8dc1-1a71359a00ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:14.875144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd05fd2-96cf-43c8-8dc1-1a71359a00ca None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:14.945651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7a6c5-8ab4-4b5a-819a-6d7061d59939 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:14.978674 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 324/657] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:05:14.979469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd05fd2-96cf-43c8-8dc1-1a71359a00ca None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:14.984497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bc35c-48c0-48b8-ac37-bdfb5c8e26d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:14.985448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bc35c-48c0-48b8-ac37-bdfb5c8e26d4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:14.985924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bc35c-48c0-48b8-ac37-bdfb5c8e26d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:14.986391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bc35c-48c0-48b8-ac37-bdfb5c8e26d4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:14.995158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bc35c-48c0-48b8-ac37-bdfb5c8e26d4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:14.997167 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95bc35c-48c0-48b8-ac37-bdfb5c8e26d4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d95bc35c-48c0-48b8-ac37-bdfb5c8e26d4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:15.010515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:15.011488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:15.012442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:15.012442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:15.012442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:15.012442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:15.012442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:15.012442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:15.012442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:15.012442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:15.012442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 15:05:15.012442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:15.015029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 334/658] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:15.016051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bc35c-48c0-48b8-ac37-bdfb5c8e26d4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:15.085489 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:15.088469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd05fd2-96cf-43c8-8dc1-1a71359a00ca None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:15.123199 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:15.129022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181491c-0dbc-4bb7-a891-626071f850fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:15.129935 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181491c-0dbc-4bb7-a891-626071f850fb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:15.130478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181491c-0dbc-4bb7-a891-626071f850fb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:15.130919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181491c-0dbc-4bb7-a891-626071f850fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:15.133531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bc35c-48c0-48b8-ac37-bdfb5c8e26d4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:15.170070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c6a96-1982-40b0-a628-9ffb2ea4d26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:15.170855 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c6a96-1982-40b0-a628-9ffb2ea4d26a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:15.171198 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c6a96-1982-40b0-a628-9ffb2ea4d26a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:15.171528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c6a96-1982-40b0-a628-9ffb2ea4d26a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:15.183311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c6a96-1982-40b0-a628-9ffb2ea4d26a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:15.185179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c6a96-1982-40b0-a628-9ffb2ea4d26a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-629c6a96-1982-40b0-a628-9ffb2ea4d26a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:15.200941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:15.201969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:15.202829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:15.202829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:15.202829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:15.202829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 335/659] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:15.203674 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c6a96-1982-40b0-a628-9ffb2ea4d26a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:15.207052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181491c-0dbc-4bb7-a891-626071f850fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:15.208755 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181491c-0dbc-4bb7-a891-626071f850fb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:15.307535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:15.311900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 325/660] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:05:15.313053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181491c-0dbc-4bb7-a891-626071f850fb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:15.317274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c6a96-1982-40b0-a628-9ffb2ea4d26a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:15.354313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb683a-3d69-485d-8a13-cb41204586c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:15.354844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb683a-3d69-485d-8a13-cb41204586c0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:15.355255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb683a-3d69-485d-8a13-cb41204586c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:15.355530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb683a-3d69-485d-8a13-cb41204586c0 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:15.356366 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb683a-3d69-485d-8a13-cb41204586c0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:15.356722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb683a-3d69-485d-8a13-cb41204586c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:15.358676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb683a-3d69-485d-8a13-cb41204586c0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:15.382431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 336/661] 10.222.0.75 () {56 vars in 1139 bytes} [Fri Sep 23 15:05:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:15.382431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb683a-3d69-485d-8a13-cb41204586c0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:15.383137 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:15.425007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181491c-0dbc-4bb7-a891-626071f850fb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:15.472843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff7378-e662-4e27-b09e-5ff7c751768d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:15.474342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff7378-e662-4e27-b09e-5ff7c751768d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:15.474844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff7378-e662-4e27-b09e-5ff7c751768d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:15.475405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff7378-e662-4e27-b09e-5ff7c751768d None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:15.476331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff7378-e662-4e27-b09e-5ff7c751768d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:15.476870 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff7378-e662-4e27-b09e-5ff7c751768d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:15.479511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff7378-e662-4e27-b09e-5ff7c751768d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:15.488804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb683a-3d69-485d-8a13-cb41204586c0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:15.493179 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 326/662] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:15.494872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff7378-e662-4e27-b09e-5ff7c751768d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:15.526843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17191879-660c-4977-9e20-f9b5a7d36c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:15.527487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17191879-660c-4977-9e20-f9b5a7d36c64 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:15.527763 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17191879-660c-4977-9e20-f9b5a7d36c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:15.528034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17191879-660c-4977-9e20-f9b5a7d36c64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:15.550296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17191879-660c-4977-9e20-f9b5a7d36c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:15.552196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17191879-660c-4977-9e20-f9b5a7d36c64 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-17191879-660c-4977-9e20-f9b5a7d36c64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:15.569368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:15.570367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:15.571479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:15.571479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:15.571479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:15.575344 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 337/663] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:15.575937 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:15.577572 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17191879-660c-4977-9e20-f9b5a7d36c64 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:15.619938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff7378-e662-4e27-b09e-5ff7c751768d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:15.640281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:15.660226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2392d8c-9664-4775-8e92-d57ffd1040f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:15.661226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2392d8c-9664-4775-8e92-d57ffd1040f9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:15.661723 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2392d8c-9664-4775-8e92-d57ffd1040f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:15.662202 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2392d8c-9664-4775-8e92-d57ffd1040f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:15.690103 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17191879-660c-4977-9e20-f9b5a7d36c64 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:15.726860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6f23f-521d-4163-94cf-5c39a0a5ef04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:15.727580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6f23f-521d-4163-94cf-5c39a0a5ef04 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:15.728262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6f23f-521d-4163-94cf-5c39a0a5ef04 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:15.728262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6f23f-521d-4163-94cf-5c39a0a5ef04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:15.737983 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6f23f-521d-4163-94cf-5c39a0a5ef04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:15.738607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2392d8c-9664-4775-8e92-d57ffd1040f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:15.739452 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee6f23f-521d-4163-94cf-5c39a0a5ef04 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:15.740499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2392d8c-9664-4775-8e92-d57ffd1040f9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-fee6f23f-521d-4163-94cf-5c39a0a5ef04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:15.755712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:15.756660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:15.757596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:15.757596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:15.757596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:15.757596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:15.757596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:15.757596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:15.757596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:15.757596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:15.757596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 338/664] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:15.758081 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6f23f-521d-4163-94cf-5c39a0a5ef04 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:15.812138 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:15.872137 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 327/665] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:05:15.872137 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2392d8c-9664-4775-8e92-d57ffd1040f9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:15.893564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6f23f-521d-4163-94cf-5c39a0a5ef04 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:15.912009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:15.939520 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e877a-3785-47d5-a058-0525788f883b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:15.940555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e877a-3785-47d5-a058-0525788f883b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:15.941067 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e877a-3785-47d5-a058-0525788f883b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:15.941533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e877a-3785-47d5-a058-0525788f883b None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:15.942436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e877a-3785-47d5-a058-0525788f883b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:15.942961 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e877a-3785-47d5-a058-0525788f883b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:15.944678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e877a-3785-47d5-a058-0525788f883b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:15.959457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 339/666] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:15.960736 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e877a-3785-47d5-a058-0525788f883b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:16.002321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2392d8c-9664-4775-8e92-d57ffd1040f9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:16.039550 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77ac72-3166-4e18-ae29-a1b819d7d8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:16.040224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77ac72-3166-4e18-ae29-a1b819d7d8fe None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:16.040513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77ac72-3166-4e18-ae29-a1b819d7d8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:16.040898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77ac72-3166-4e18-ae29-a1b819d7d8fe None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:16.041591 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77ac72-3166-4e18-ae29-a1b819d7d8fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:16.041896 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77ac72-3166-4e18-ae29-a1b819d7d8fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:16.043342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77ac72-3166-4e18-ae29-a1b819d7d8fe None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:16.055494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 328/667] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:16.056577 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77ac72-3166-4e18-ae29-a1b819d7d8fe None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:16.071955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e877a-3785-47d5-a058-0525788f883b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:16.079449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:16.106465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11952be5-b7bb-4fa8-ba66-0622a56b79db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:16.107091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11952be5-b7bb-4fa8-ba66-0622a56b79db None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:16.107363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11952be5-b7bb-4fa8-ba66-0622a56b79db None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:16.107626 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11952be5-b7bb-4fa8-ba66-0622a56b79db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:16.118885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11952be5-b7bb-4fa8-ba66-0622a56b79db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:16.120321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11952be5-b7bb-4fa8-ba66-0622a56b79db None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-11952be5-b7bb-4fa8-ba66-0622a56b79db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:16.134930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:16.135881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:16.136780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:16.136780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:16.136780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:16.136780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 340/668] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:16.137295 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3f6b5-ff44-48fb-801e-1cbfaec80a56 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:16.137569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3f6b5-ff44-48fb-801e-1cbfaec80a56 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:16.137830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3f6b5-ff44-48fb-801e-1cbfaec80a56 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:16.139204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 341/669] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:16.140033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d142b74-f42a-4f3e-bd38-038807790a01 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:16.140310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d142b74-f42a-4f3e-bd38-038807790a01 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:16.140760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d142b74-f42a-4f3e-bd38-038807790a01 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:16.142530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 342/670] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:16.142530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d142b74-f42a-4f3e-bd38-038807790a01 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:16.165140 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77ac72-3166-4e18-ae29-a1b819d7d8fe None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:16.171173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:16.203370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8270c-7f6a-4fdd-a7b9-44b71c85c2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:16.203995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8270c-7f6a-4fdd-a7b9-44b71c85c2bf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:16.204270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8270c-7f6a-4fdd-a7b9-44b71c85c2bf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:16.204532 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8270c-7f6a-4fdd-a7b9-44b71c85c2bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:16.211189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8270c-7f6a-4fdd-a7b9-44b71c85c2bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:16.212683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8270c-7f6a-4fdd-a7b9-44b71c85c2bf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-b3a8270c-7f6a-4fdd-a7b9-44b71c85c2bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:16.223624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:16.224711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:16.230711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:16.230711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:16.230711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:16.230711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:16.230711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:16.230711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:16.230711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:16.230711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:16.230711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 329/671] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:16.230711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8270c-7f6a-4fdd-a7b9-44b71c85c2bf None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:16.250082 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d142b74-f42a-4f3e-bd38-038807790a01 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:16.282575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264dc69a-e834-4cad-9428-9a8b60a54e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:16.283202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dc69a-e834-4cad-9428-9a8b60a54e36 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:16.283472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dc69a-e834-4cad-9428-9a8b60a54e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:16.283741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264dc69a-e834-4cad-9428-9a8b60a54e36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:16.284474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264dc69a-e834-4cad-9428-9a8b60a54e36 None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=1cd51776db3d426ea1bf1133344fa187) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:16.284806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264dc69a-e834-4cad-9428-9a8b60a54e36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:16.286223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264dc69a-e834-4cad-9428-9a8b60a54e36 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:16.292286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 343/672] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:16 2022] GET /identity/v3/role_assignments?user.id=1cd51776db3d426ea1bf1133344fa187&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:16.293441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00327830-e389-4f09-9357-286aa473c705 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:16.293717 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00327830-e389-4f09-9357-286aa473c705 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:16.293980 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00327830-e389-4f09-9357-286aa473c705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:16.316678 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:16.320912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-00327830-e389-4f09-9357-286aa473c705 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:16.322690 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-00327830-e389-4f09-9357-286aa473c705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:16.338150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8270c-7f6a-4fdd-a7b9-44b71c85c2bf None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:16.382480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010e15c-b0c3-4058-922e-596fc0d1636b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:16.383423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010e15c-b0c3-4058-922e-596fc0d1636b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:16.384076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010e15c-b0c3-4058-922e-596fc0d1636b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:16.385337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010e15c-b0c3-4058-922e-596fc0d1636b None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:16.386596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010e15c-b0c3-4058-922e-596fc0d1636b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:16.387073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010e15c-b0c3-4058-922e-596fc0d1636b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:16.388671 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010e15c-b0c3-4058-922e-596fc0d1636b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:16.403209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 344/673] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:16.404579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf05a7-36a5-42ae-8544-77720eec2147 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:16.404883 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf05a7-36a5-42ae-8544-77720eec2147 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:16.405144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf05a7-36a5-42ae-8544-77720eec2147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:16.409967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 330/674] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:16.409967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010e15c-b0c3-4058-922e-596fc0d1636b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:16.429288 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-89bf05a7-36a5-42ae-8544-77720eec2147 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:16.431653 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-89bf05a7-36a5-42ae-8544-77720eec2147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:16.449274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:16.522853 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 345/675] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:16.523738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf05a7-36a5-42ae-8544-77720eec2147 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:16.547891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010e15c-b0c3-4058-922e-596fc0d1636b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:16.591864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5f322-8575-4144-86fd-ecea41f79cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:16.592923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5f322-8575-4144-86fd-ecea41f79cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:16.593432 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5f322-8575-4144-86fd-ecea41f79cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:16.593875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5f322-8575-4144-86fd-ecea41f79cd9 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:16.594853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5f322-8575-4144-86fd-ecea41f79cd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:16.595332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5f322-8575-4144-86fd-ecea41f79cd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:16.597053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5f322-8575-4144-86fd-ecea41f79cd9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:16.607908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:16.616106 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 331/676] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:16.623477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5f322-8575-4144-86fd-ecea41f79cd9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:16.662012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf05a7-36a5-42ae-8544-77720eec2147 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:16.719205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702805e-3953-4275-bacd-fbbb23f4bd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:16.719848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702805e-3953-4275-bacd-fbbb23f4bd06 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:16.720120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702805e-3953-4275-bacd-fbbb23f4bd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:16.720383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702805e-3953-4275-bacd-fbbb23f4bd06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:16.730306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702805e-3953-4275-bacd-fbbb23f4bd06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:16.731758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e702805e-3953-4275-bacd-fbbb23f4bd06 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:16.747115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5f322-8575-4144-86fd-ecea41f79cd9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-e702805e-3953-4275-bacd-fbbb23f4bd06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:16.747485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:16.748891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:16.750358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:16.750358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:16.750358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:16.750358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:16.750358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:16.750358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:16.750358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:16.750358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:16.750358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 346/677] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:16.751152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702805e-3953-4275-bacd-fbbb23f4bd06 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:16.773260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:16.792553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb78c7-235f-48d8-86a3-5f12bde7dcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:16.793198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb78c7-235f-48d8-86a3-5f12bde7dcdc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:16.793649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb78c7-235f-48d8-86a3-5f12bde7dcdc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:16.794036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb78c7-235f-48d8-86a3-5f12bde7dcdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:16.794837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb78c7-235f-48d8-86a3-5f12bde7dcdc None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=2af1ba390fff41a68ff1a0035901c318) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:16.795135 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb78c7-235f-48d8-86a3-5f12bde7dcdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:16.796857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb78c7-235f-48d8-86a3-5f12bde7dcdc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:16.804449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 332/678] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:16 2022] GET /identity/v3/role_assignments?user.id=2af1ba390fff41a68ff1a0035901c318&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:16.805311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb78c7-235f-48d8-86a3-5f12bde7dcdc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:16.917861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:16.918506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702805e-3953-4275-bacd-fbbb23f4bd06 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:16.957870 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c7d68-0a07-40c2-9fd5-8aff9629f5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:16.958506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c7d68-0a07-40c2-9fd5-8aff9629f5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:16.958774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c7d68-0a07-40c2-9fd5-8aff9629f5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:16.959036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c7d68-0a07-40c2-9fd5-8aff9629f5f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:16.971973 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb78c7-235f-48d8-86a3-5f12bde7dcdc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:16.975894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c7d68-0a07-40c2-9fd5-8aff9629f5f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:16.977828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c7d68-0a07-40c2-9fd5-8aff9629f5f0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-1f9c7d68-0a07-40c2-9fd5-8aff9629f5f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:17.000830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:17.001811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:17.002677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:17.002677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:17.002677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:17.009912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 347/679] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:17.010104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c7d68-0a07-40c2-9fd5-8aff9629f5f0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:17.020321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05024654-d6d6-4b68-8e6c-ab58a217f60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:17.020985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05024654-d6d6-4b68-8e6c-ab58a217f60f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:17.021262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05024654-d6d6-4b68-8e6c-ab58a217f60f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:17.021529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05024654-d6d6-4b68-8e6c-ab58a217f60f None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:17.022237 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05024654-d6d6-4b68-8e6c-ab58a217f60f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:17.023765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05024654-d6d6-4b68-8e6c-ab58a217f60f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:17.025595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05024654-d6d6-4b68-8e6c-ab58a217f60f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:17.041784 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 333/680] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:17.042666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05024654-d6d6-4b68-8e6c-ab58a217f60f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:17.092700 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:17.136557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c7d68-0a07-40c2-9fd5-8aff9629f5f0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:17.172212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05024654-d6d6-4b68-8e6c-ab58a217f60f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:17.193419 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f91daa-4c07-4c14-819e-2877340dfdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:17.194626 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f91daa-4c07-4c14-819e-2877340dfdad None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:17.194928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f91daa-4c07-4c14-819e-2877340dfdad None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:17.195202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f91daa-4c07-4c14-819e-2877340dfdad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:17.199828 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:17.213384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f91daa-4c07-4c14-819e-2877340dfdad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:17.217819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544b460-514a-44b9-8995-dc6a4079fb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:17.218712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544b460-514a-44b9-8995-dc6a4079fb00 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:17.219199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544b460-514a-44b9-8995-dc6a4079fb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-79f91daa-4c07-4c14-819e-2877340dfdad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:17.219427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:17.220298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:17.221165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:17.221165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:17.221165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:17.221165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:17.221165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:17.221165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:17.221165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:17.221165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:17.221165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:17.221165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:17.221165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:17.221165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:17.221824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 348/681] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:17.222638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544b460-514a-44b9-8995-dc6a4079fb00 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:17.222828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f91daa-4c07-4c14-819e-2877340dfdad None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:17.227608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544b460-514a-44b9-8995-dc6a4079fb00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:17.228192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544b460-514a-44b9-8995-dc6a4079fb00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:17.230408 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544b460-514a-44b9-8995-dc6a4079fb00 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:17.244535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 334/682] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:17.245616 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544b460-514a-44b9-8995-dc6a4079fb00 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:17.360040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f91daa-4c07-4c14-819e-2877340dfdad None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:17.366722 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544b460-514a-44b9-8995-dc6a4079fb00 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:17.392952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:17.409747 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0058e6-3ea4-4966-8ffb-e95ca2885292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:17.410748 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0058e6-3ea4-4966-8ffb-e95ca2885292 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:17.411089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0058e6-3ea4-4966-8ffb-e95ca2885292 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:17.411422 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0058e6-3ea4-4966-8ffb-e95ca2885292 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:17.412240 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0058e6-3ea4-4966-8ffb-e95ca2885292 None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=02afdedf8d394bbfa977ab7a22fafea7) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:17.412548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0058e6-3ea4-4966-8ffb-e95ca2885292 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:17.414116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0058e6-3ea4-4966-8ffb-e95ca2885292 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:17.416101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682343bf-da04-4528-ab6f-312c83574a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:17.417144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682343bf-da04-4528-ab6f-312c83574a8e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:17.418135 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682343bf-da04-4528-ab6f-312c83574a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:17.418633 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682343bf-da04-4528-ab6f-312c83574a8e None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:17.419780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682343bf-da04-4528-ab6f-312c83574a8e None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:17.420284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682343bf-da04-4528-ab6f-312c83574a8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:17.421447 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 349/683] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:17 2022] GET /identity/v3/role_assignments?user.id=02afdedf8d394bbfa977ab7a22fafea7&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:17.422296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682343bf-da04-4528-ab6f-312c83574a8e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:17.422405 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0058e6-3ea4-4966-8ffb-e95ca2885292 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:17.432752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 335/684] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:17.441765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682343bf-da04-4528-ab6f-312c83574a8e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:17.564507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:17.580321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0058e6-3ea4-4966-8ffb-e95ca2885292 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:17.645973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db0983-0561-4182-be1d-8c9aa67f2132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:17.647856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db0983-0561-4182-be1d-8c9aa67f2132 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:17.648475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db0983-0561-4182-be1d-8c9aa67f2132 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:17.650631 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db0983-0561-4182-be1d-8c9aa67f2132 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:17.651757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db0983-0561-4182-be1d-8c9aa67f2132 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:17.658228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db0983-0561-4182-be1d-8c9aa67f2132 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:17.673184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682343bf-da04-4528-ab6f-312c83574a8e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:17.690673 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 350/685] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:05:17 2022] POST /identity/v3/users => generated 254 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:05:17.723493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5d8d1-29bf-4efe-a76f-17edcd50a8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:17.724491 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5d8d1-29bf-4efe-a76f-17edcd50a8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:17.725022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5d8d1-29bf-4efe-a76f-17edcd50a8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:17.725492 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5d8d1-29bf-4efe-a76f-17edcd50a8e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:17.739864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5d8d1-29bf-4efe-a76f-17edcd50a8e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:17.741737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5d8d1-29bf-4efe-a76f-17edcd50a8e3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-bac5d8d1-29bf-4efe-a76f-17edcd50a8e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:17.761134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:17.762196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:17.763065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:17.763065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:17.763065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:17.765289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 336/686] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:17.771553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db0983-0561-4182-be1d-8c9aa67f2132 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:17.826056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:17.902027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db0983-0561-4182-be1d-8c9aa67f2132 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:17.942197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b93392-2236-4dbb-9b6c-3fc3d5a78552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:17.942815 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b93392-2236-4dbb-9b6c-3fc3d5a78552 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:17.943053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b93392-2236-4dbb-9b6c-3fc3d5a78552 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:17.943250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b93392-2236-4dbb-9b6c-3fc3d5a78552 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:17.943907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b93392-2236-4dbb-9b6c-3fc3d5a78552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:17.944203 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b93392-2236-4dbb-9b6c-3fc3d5a78552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:17.945652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b93392-2236-4dbb-9b6c-3fc3d5a78552 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:17.959497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 351/687] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:17.965812 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5d8d1-29bf-4efe-a76f-17edcd50a8e3 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:18.075010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:18.079752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5d8d1-29bf-4efe-a76f-17edcd50a8e3 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:18.121731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e3cd6-040a-4c35-935d-dbff1043dc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:18.122371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e3cd6-040a-4c35-935d-dbff1043dc34 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:18.122643 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e3cd6-040a-4c35-935d-dbff1043dc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:18.122910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e3cd6-040a-4c35-935d-dbff1043dc34 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:18.131120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e3cd6-040a-4c35-935d-dbff1043dc34 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:18.132651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e3cd6-040a-4c35-935d-dbff1043dc34 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-854e3cd6-040a-4c35-935d-dbff1043dc34 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:18.145168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:18.146144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:18.147090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:18.147090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:18.147090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:18.147090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:18.147090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:18.147090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:18.147090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:18.147090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:18.147090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 15:05:18.147090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:18.147090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 337/688] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:05:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:18.153126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b93392-2236-4dbb-9b6c-3fc3d5a78552 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:18.277317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b93392-2236-4dbb-9b6c-3fc3d5a78552 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:18.278850 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:18.329899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad63212-bc7a-47a1-a960-511b19b0efd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:18.330528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad63212-bc7a-47a1-a960-511b19b0efd7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:18.330806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad63212-bc7a-47a1-a960-511b19b0efd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:18.331074 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad63212-bc7a-47a1-a960-511b19b0efd7 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:18.331821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad63212-bc7a-47a1-a960-511b19b0efd7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:18.332118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad63212-bc7a-47a1-a960-511b19b0efd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:18.333632 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad63212-bc7a-47a1-a960-511b19b0efd7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:18.353956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 352/689] 10.222.0.75 () {56 vars in 1135 bytes} [Fri Sep 23 15:05:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:18.359870 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e3cd6-040a-4c35-935d-dbff1043dc34 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:18.489773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e3cd6-040a-4c35-935d-dbff1043dc34 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:18.536083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f5c7d-6a67-467a-aaab-4a970ee922a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:18.536624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f5c7d-6a67-467a-aaab-4a970ee922a3 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:18.536903 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f5c7d-6a67-467a-aaab-4a970ee922a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:18.537168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f5c7d-6a67-467a-aaab-4a970ee922a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:18.584295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:18.629994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f5c7d-6a67-467a-aaab-4a970ee922a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:18.631451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f5c7d-6a67-467a-aaab-4a970ee922a3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:18.757595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 338/690] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:05:18.764091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad63212-bc7a-47a1-a960-511b19b0efd7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:18.813054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163db11a-44a7-4484-b9f3-b41f9b509429 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:18.813374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163db11a-44a7-4484-b9f3-b41f9b509429 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:18.813594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163db11a-44a7-4484-b9f3-b41f9b509429 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:18.816625 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 339/691] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:18.836613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487076b1-de73-46e2-90cf-f050e2730314 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:18.836613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487076b1-de73-46e2-90cf-f050e2730314 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:18.836613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487076b1-de73-46e2-90cf-f050e2730314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:18.859212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-487076b1-de73-46e2-90cf-f050e2730314 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:18.860401 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-487076b1-de73-46e2-90cf-f050e2730314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:18.939240 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad63212-bc7a-47a1-a960-511b19b0efd7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:18.951267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:18.974254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 340/692] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:05:18.994018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c047787-13db-42a8-aba4-b14dc2fb28a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:18.994723 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c047787-13db-42a8-aba4-b14dc2fb28a1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:18.994907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c047787-13db-42a8-aba4-b14dc2fb28a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:18.995173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c047787-13db-42a8-aba4-b14dc2fb28a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:19.005651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c047787-13db-42a8-aba4-b14dc2fb28a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:19.007080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c047787-13db-42a8-aba4-b14dc2fb28a1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-6c047787-13db-42a8-aba4-b14dc2fb28a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:19.022783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:19.023785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:19.024977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:19.024977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:19.024977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:19.024977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 353/693] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:19.030154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487076b1-de73-46e2-90cf-f050e2730314 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:19.160970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487076b1-de73-46e2-90cf-f050e2730314 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:19.208350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b362179-d81d-44f3-b557-67a56e806a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:19.209018 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b362179-d81d-44f3-b557-67a56e806a9e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:19.209285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b362179-d81d-44f3-b557-67a56e806a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:19.209590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b362179-d81d-44f3-b557-67a56e806a9e None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:19.210248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b362179-d81d-44f3-b557-67a56e806a9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:19.210537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b362179-d81d-44f3-b557-67a56e806a9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:19.211927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b362179-d81d-44f3-b557-67a56e806a9e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:19.227375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 341/694] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:19.233797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c047787-13db-42a8-aba4-b14dc2fb28a1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:19.249871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:19.275040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b362179-d81d-44f3-b557-67a56e806a9e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:19.365013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:19.417517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b362179-d81d-44f3-b557-67a56e806a9e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:19.420605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c047787-13db-42a8-aba4-b14dc2fb28a1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:19.467781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c05d94-a84d-43c8-89f4-e9f458250fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:19.468810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c05d94-a84d-43c8-89f4-e9f458250fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:19.469283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c05d94-a84d-43c8-89f4-e9f458250fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:19.469739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c05d94-a84d-43c8-89f4-e9f458250fd5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:19.470510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c05d94-a84d-43c8-89f4-e9f458250fd5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:19.472467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c05d94-a84d-43c8-89f4-e9f458250fd5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-88c05d94-a84d-43c8-89f4-e9f458250fd5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:19.487228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:19.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 15:05:19.489207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:19.492381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 342/695] 10.222.0.75 () {56 vars in 1129 bytes} [Fri Sep 23 15:05:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:19.500909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c05d94-a84d-43c8-89f4-e9f458250fd5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:19.514007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4190f6-fb2e-4df2-a05a-1590436d2452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:19.514769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4190f6-fb2e-4df2-a05a-1590436d2452 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:19.515127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4190f6-fb2e-4df2-a05a-1590436d2452 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:19.515472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4190f6-fb2e-4df2-a05a-1590436d2452 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:19.525456 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4190f6-fb2e-4df2-a05a-1590436d2452 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:19.527231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4190f6-fb2e-4df2-a05a-1590436d2452 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-8e4190f6-fb2e-4df2-a05a-1590436d2452 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:19.545261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:19.546237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:19.547617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:19.547617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:19.547617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:19.547617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:19.547617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:19.547617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:19.547617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:19.547617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:19.550077 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 354/696] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:19.551568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c4d59-c364-4fa4-bf54-536924255e50 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:19.552062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c4d59-c364-4fa4-bf54-536924255e50 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:19.552528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c4d59-c364-4fa4-bf54-536924255e50 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:19.554594 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 355/697] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:19.555602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c4d59-c364-4fa4-bf54-536924255e50 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:19.638371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c05d94-a84d-43c8-89f4-e9f458250fd5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:19.650841 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:19.656087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:19.681365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fa32e-12c4-4e6b-909d-fae4e4a5d51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:19.682205 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fa32e-12c4-4e6b-909d-fae4e4a5d51b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:19.682786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fa32e-12c4-4e6b-909d-fae4e4a5d51b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:19.683216 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fa32e-12c4-4e6b-909d-fae4e4a5d51b None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:19.684082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fa32e-12c4-4e6b-909d-fae4e4a5d51b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:19.684600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fa32e-12c4-4e6b-909d-fae4e4a5d51b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:19.686645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fa32e-12c4-4e6b-909d-fae4e4a5d51b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:19.690274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c4d59-c364-4fa4-bf54-536924255e50 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:19.697223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 343/698] 10.222.0.75 () {56 vars in 1124 bytes} [Fri Sep 23 15:05:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:19.698969 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b394b8b-ee95-49ab-bc91-9965cbfaa97f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:19.699473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b394b8b-ee95-49ab-bc91-9965cbfaa97f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:19.699941 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b394b8b-ee95-49ab-bc91-9965cbfaa97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:19.721188 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-3b394b8b-ee95-49ab-bc91-9965cbfaa97f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:19.722884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3b394b8b-ee95-49ab-bc91-9965cbfaa97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:19.729366 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527f8c3-0424-4031-98d2-d10890fda112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:19.729977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527f8c3-0424-4031-98d2-d10890fda112 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:19.730290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527f8c3-0424-4031-98d2-d10890fda112 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:19.730561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527f8c3-0424-4031-98d2-d10890fda112 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:19.731290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527f8c3-0424-4031-98d2-d10890fda112 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:19.731586 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527f8c3-0424-4031-98d2-d10890fda112 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:19.733395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0527f8c3-0424-4031-98d2-d10890fda112 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:19.754326 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 356/699] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:19.756176 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527f8c3-0424-4031-98d2-d10890fda112 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:19.816711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 344/700] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:19.817990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b394b8b-ee95-49ab-bc91-9965cbfaa97f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:19.895565 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527f8c3-0424-4031-98d2-d10890fda112 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:19.949639 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:19.961901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b394b8b-ee95-49ab-bc91-9965cbfaa97f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:19.965275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccedd2-739d-4cdb-b24d-b2a212328e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:19.965906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccedd2-739d-4cdb-b24d-b2a212328e6a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:19.966181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccedd2-739d-4cdb-b24d-b2a212328e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:19.966451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccedd2-739d-4cdb-b24d-b2a212328e6a None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:19.967101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccedd2-739d-4cdb-b24d-b2a212328e6a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:19.967408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccedd2-739d-4cdb-b24d-b2a212328e6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:19.968805 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccedd2-739d-4cdb-b24d-b2a212328e6a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:19.971600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:19.979703 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 357/701] 10.222.0.75 () {56 vars in 1124 bytes} [Fri Sep 23 15:05:19 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:20.014655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95615dec-f048-4010-9cb6-2a474223d7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:20.015657 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95615dec-f048-4010-9cb6-2a474223d7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:20.016171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95615dec-f048-4010-9cb6-2a474223d7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:20.016668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95615dec-f048-4010-9cb6-2a474223d7e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:20.017627 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95615dec-f048-4010-9cb6-2a474223d7e6 None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=a8614f90cebb4a4eaf7ab5512927fb3c) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:20.018207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95615dec-f048-4010-9cb6-2a474223d7e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:20.019883 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95615dec-f048-4010-9cb6-2a474223d7e6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:20.027684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 345/702] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:19 2022] GET /identity/v3/role_assignments?user.id=a8614f90cebb4a4eaf7ab5512927fb3c&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:20.195449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccedd2-739d-4cdb-b24d-b2a212328e6a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:20.314580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccedd2-739d-4cdb-b24d-b2a212328e6a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:20.345625 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a174870-72f7-4391-8058-a6fed41f361c None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:20.346250 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a174870-72f7-4391-8058-a6fed41f361c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:20.346753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a174870-72f7-4391-8058-a6fed41f361c None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:20.348306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 346/703] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:05:20.354621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434f12b-6a26-4912-b87c-6e3b27eddfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:20.355258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434f12b-6a26-4912-b87c-6e3b27eddfda None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:20.355537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434f12b-6a26-4912-b87c-6e3b27eddfda None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:20.355804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434f12b-6a26-4912-b87c-6e3b27eddfda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:20.365864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd491a3-5f33-4e7c-95c9-7353d657d9ae None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:20.366137 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd491a3-5f33-4e7c-95c9-7353d657d9ae None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:20.366403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd491a3-5f33-4e7c-95c9-7353d657d9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:20.371805 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434f12b-6a26-4912-b87c-6e3b27eddfda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-0434f12b-6a26-4912-b87c-6e3b27eddfda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:20.379139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:20.380088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:20.381042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:20.381042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:20.381042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:20.381042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:20.381042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:20.381042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:20.381042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:20.381042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:20.381042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:20.381042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:20.381042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:05:20.381042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:20.382652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 358/704] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:05:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:20.388372 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434f12b-6a26-4912-b87c-6e3b27eddfda None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:20.389748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-dfd491a3-5f33-4e7c-95c9-7353d657d9ae None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:20.391017 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dfd491a3-5f33-4e7c-95c9-7353d657d9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:20.470801 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:20.474711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 347/705] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 15:05:20.530535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434f12b-6a26-4912-b87c-6e3b27eddfda None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:20.567641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024fa96-1c98-454a-ae8d-78ff71fde5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:20.568356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024fa96-1c98-454a-ae8d-78ff71fde5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:20.568780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024fa96-1c98-454a-ae8d-78ff71fde5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:20.569185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024fa96-1c98-454a-ae8d-78ff71fde5f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:20.569962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024fa96-1c98-454a-ae8d-78ff71fde5f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:20.570378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024fa96-1c98-454a-ae8d-78ff71fde5f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:20.572038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024fa96-1c98-454a-ae8d-78ff71fde5f2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:20.582182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 359/706] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:05:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:20.588016 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd491a3-5f33-4e7c-95c9-7353d657d9ae None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:20.695158 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:20.698899 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd491a3-5f33-4e7c-95c9-7353d657d9ae None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:20.735542 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea06a53-e6d3-4dad-b36d-9ba7bae41c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:20.736178 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea06a53-e6d3-4dad-b36d-9ba7bae41c07 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:20.736450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea06a53-e6d3-4dad-b36d-9ba7bae41c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:20.736748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea06a53-e6d3-4dad-b36d-9ba7bae41c07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:20.747598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea06a53-e6d3-4dad-b36d-9ba7bae41c07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:20.749168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea06a53-e6d3-4dad-b36d-9ba7bae41c07 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:20.764930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024fa96-1c98-454a-ae8d-78ff71fde5f2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-2ea06a53-e6d3-4dad-b36d-9ba7bae41c07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:20.765675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:20.766574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:20.767473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:20.767473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:20.767473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:20.767473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 348/707] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:20.786455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea06a53-e6d3-4dad-b36d-9ba7bae41c07 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:20.887355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024fa96-1c98-454a-ae8d-78ff71fde5f2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:20.906674 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:20.926463 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec841b-ad0c-4c71-b802-af5da6db5c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:20.927107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec841b-ad0c-4c71-b802-af5da6db5c07 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:20.927385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec841b-ad0c-4c71-b802-af5da6db5c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:20.927656 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec841b-ad0c-4c71-b802-af5da6db5c07 None admin] PATH_INFO: `/v3/services/network` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:20.928248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec841b-ad0c-4c71-b802-af5da6db5c07 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:20.929744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec841b-ad0c-4c71-b802-af5da6db5c07 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-feec841b-ad0c-4c71-b802-af5da6db5c07 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:20.941824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:20.942833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:20.943685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 360/708] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:20 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:20.953821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea06a53-e6d3-4dad-b36d-9ba7bae41c07 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:20.958171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec841b-ad0c-4c71-b802-af5da6db5c07 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:21.003132 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffda60-a1b1-4a88-8a17-02f8f7c346f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:21.004807 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffda60-a1b1-4a88-8a17-02f8f7c346f6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:21.005282 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffda60-a1b1-4a88-8a17-02f8f7c346f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:21.005787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffda60-a1b1-4a88-8a17-02f8f7c346f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:21.006886 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffda60-a1b1-4a88-8a17-02f8f7c346f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:21.008237 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffda60-a1b1-4a88-8a17-02f8f7c346f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:21.010058 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffda60-a1b1-4a88-8a17-02f8f7c346f6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:21.026772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 349/709] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:21.028740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66d1fb-a941-466b-96f1-06a24b82bc14 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:21.029258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66d1fb-a941-466b-96f1-06a24b82bc14 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:21.029754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66d1fb-a941-466b-96f1-06a24b82bc14 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:21.031442 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 350/710] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:21.035575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66d1fb-a941-466b-96f1-06a24b82bc14 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:21.055088 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:21.092895 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec841b-ad0c-4c71-b802-af5da6db5c07 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:21.133407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad915c4e-a914-4cc3-85a3-9085f512002d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:21.134587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad915c4e-a914-4cc3-85a3-9085f512002d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:21.135053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad915c4e-a914-4cc3-85a3-9085f512002d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:21.135810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad915c4e-a914-4cc3-85a3-9085f512002d None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:21.137590 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad915c4e-a914-4cc3-85a3-9085f512002d None admin] RBAC: Adding query filter params (name=network) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:21.138199 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad915c4e-a914-4cc3-85a3-9085f512002d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:21.139912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad915c4e-a914-4cc3-85a3-9085f512002d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:21.145842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 361/711] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Sep 23 15:05:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:21.147754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577e91e-1b13-4b8b-a911-50aa802a028e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:21.148354 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577e91e-1b13-4b8b-a911-50aa802a028e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:21.148852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577e91e-1b13-4b8b-a911-50aa802a028e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:21.168347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3577e91e-1b13-4b8b-a911-50aa802a028e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:21.168539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66d1fb-a941-466b-96f1-06a24b82bc14 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:21.170092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3577e91e-1b13-4b8b-a911-50aa802a028e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:21.207269 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec6e44-6b65-4212-890d-2e8cd90594bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:21.207899 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec6e44-6b65-4212-890d-2e8cd90594bf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:21.208172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec6e44-6b65-4212-890d-2e8cd90594bf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:21.208440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec6e44-6b65-4212-890d-2e8cd90594bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:21.216218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec6e44-6b65-4212-890d-2e8cd90594bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:21.217681 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec6e44-6b65-4212-890d-2e8cd90594bf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:21.218313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-5dec6e44-6b65-4212-890d-2e8cd90594bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:21.234028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:21.235083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:21.236039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:21.236039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:21.236039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:21.236039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:21.236039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:21.236039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:21.236039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:21.236039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:21.236039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:05:21.236039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:21.240228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 351/712] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Sep 23 15:05:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:21.241474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec6e44-6b65-4212-890d-2e8cd90594bf None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:21.256786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 362/713] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:21.258021 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577e91e-1b13-4b8b-a911-50aa802a028e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:21.318449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:21.354691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec6e44-6b65-4212-890d-2e8cd90594bf None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:21.392559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00fe39-432b-48c9-b0a6-51bd714dfbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:21.393111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00fe39-432b-48c9-b0a6-51bd714dfbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:21.393602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00fe39-432b-48c9-b0a6-51bd714dfbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:21.393885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00fe39-432b-48c9-b0a6-51bd714dfbb8 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:21.394525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00fe39-432b-48c9-b0a6-51bd714dfbb8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:21.394826 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00fe39-432b-48c9-b0a6-51bd714dfbb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:21.396285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00fe39-432b-48c9-b0a6-51bd714dfbb8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:21.402794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 352/714] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Sep 23 15:05:21 2022] GET /identity/v3/services?type=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:21.422446 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577e91e-1b13-4b8b-a911-50aa802a028e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:21.461685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff392f8-a2d3-4aa3-9f57-d0df2b9caa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:21.462698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff392f8-a2d3-4aa3-9f57-d0df2b9caa15 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:21.463333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff392f8-a2d3-4aa3-9f57-d0df2b9caa15 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:21.463789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff392f8-a2d3-4aa3-9f57-d0df2b9caa15 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:21.464752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff392f8-a2d3-4aa3-9f57-d0df2b9caa15 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:21.465284 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff392f8-a2d3-4aa3-9f57-d0df2b9caa15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:21.466970 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff392f8-a2d3-4aa3-9f57-d0df2b9caa15 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:21.487894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 363/715] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Sep 23 15:05:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:21.493607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00fe39-432b-48c9-b0a6-51bd714dfbb8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:21.568931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff392f8-a2d3-4aa3-9f57-d0df2b9caa15 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:21.646985 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:21.662882 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00fe39-432b-48c9-b0a6-51bd714dfbb8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:21.705253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f768f-d658-467f-b299-9c3b14fd6f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:21.705909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f768f-d658-467f-b299-9c3b14fd6f45 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:21.706127 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f768f-d658-467f-b299-9c3b14fd6f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:21.706380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f768f-d658-467f-b299-9c3b14fd6f45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:21.711301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:21.751244 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff392f8-a2d3-4aa3-9f57-d0df2b9caa15 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:21.795808 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345f8df-fbe2-4dce-9003-5a9a2a2ba615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:21.796450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345f8df-fbe2-4dce-9003-5a9a2a2ba615 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:21.796757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345f8df-fbe2-4dce-9003-5a9a2a2ba615 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:21.797026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345f8df-fbe2-4dce-9003-5a9a2a2ba615 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:21.810414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f768f-d658-467f-b299-9c3b14fd6f45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:21.811864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f768f-d658-467f-b299-9c3b14fd6f45 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:21.824070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8345f8df-fbe2-4dce-9003-5a9a2a2ba615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-8345f8df-fbe2-4dce-9003-5a9a2a2ba615 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:21.833768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:21.834767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:21.835603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:21.835603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:21.835603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:21.835603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:21.835603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:21.835603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:21.835603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:21.835603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:21.835603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:21.835603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:21.835603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:21.835603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:21.836235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 364/716] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:21.844364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345f8df-fbe2-4dce-9003-5a9a2a2ba615 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:21.961645 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:21.970990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345f8df-fbe2-4dce-9003-5a9a2a2ba615 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:22.011407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d6175-6509-4d46-b482-2dfb7235f22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:22.012182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d6175-6509-4d46-b482-2dfb7235f22a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:22.012462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d6175-6509-4d46-b482-2dfb7235f22a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:22.012765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d6175-6509-4d46-b482-2dfb7235f22a None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:22.013443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d6175-6509-4d46-b482-2dfb7235f22a None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:22.013744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d6175-6509-4d46-b482-2dfb7235f22a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:22.015135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d6175-6509-4d46-b482-2dfb7235f22a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:22.025041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 365/717] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:22.031774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d6175-6509-4d46-b482-2dfb7235f22a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:22.095683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 353/718] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:05:22.114338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f768f-d658-467f-b299-9c3b14fd6f45 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:22.166387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d6175-6509-4d46-b482-2dfb7235f22a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:22.182105 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:22.208233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07055a8-e285-4c12-8d1d-25303e3e8394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:22.208908 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07055a8-e285-4c12-8d1d-25303e3e8394 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:22.209185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07055a8-e285-4c12-8d1d-25303e3e8394 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:22.209458 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07055a8-e285-4c12-8d1d-25303e3e8394 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:22.222006 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07055a8-e285-4c12-8d1d-25303e3e8394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:22.223397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07055a8-e285-4c12-8d1d-25303e3e8394 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-c07055a8-e285-4c12-8d1d-25303e3e8394 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:22.240629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:22.241630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:22.242527 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:22.242527 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:22.242527 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:22.242527 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 366/719] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:22.248431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07055a8-e285-4c12-8d1d-25303e3e8394 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:22.262368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:22.331359 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f768f-d658-467f-b299-9c3b14fd6f45 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:22.359077 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07055a8-e285-4c12-8d1d-25303e3e8394 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:22.367514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fcb18-c52e-4ffb-bfb1-495ec2262820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:22.368235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fcb18-c52e-4ffb-bfb1-495ec2262820 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:22.368542 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fcb18-c52e-4ffb-bfb1-495ec2262820 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:22.368879 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fcb18-c52e-4ffb-bfb1-495ec2262820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:22.379222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fcb18-c52e-4ffb-bfb1-495ec2262820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:22.380700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fcb18-c52e-4ffb-bfb1-495ec2262820 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-c32fcb18-c52e-4ffb-bfb1-495ec2262820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:22.402792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:22.403790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:22.404694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:22.404694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:22.404694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:22.410923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a451a-3662-4cca-a8bc-6bbe34df9a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:22.411553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a451a-3662-4cca-a8bc-6bbe34df9a6b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:22.411831 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a451a-3662-4cca-a8bc-6bbe34df9a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:22.412107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a451a-3662-4cca-a8bc-6bbe34df9a6b None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:22.412796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a451a-3662-4cca-a8bc-6bbe34df9a6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:22.413100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a451a-3662-4cca-a8bc-6bbe34df9a6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:22.414561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a451a-3662-4cca-a8bc-6bbe34df9a6b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:22.417277 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 354/720] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:22.423865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fcb18-c52e-4ffb-bfb1-495ec2262820 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:22.455219 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 367/721] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:22.456193 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a451a-3662-4cca-a8bc-6bbe34df9a6b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:22.595208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:22.605133 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:22.610562 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fcb18-c52e-4ffb-bfb1-495ec2262820 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:22.652570 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6844256-a8f6-459b-b636-0564339e9c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:22.653752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6844256-a8f6-459b-b636-0564339e9c9b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:22.654244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6844256-a8f6-459b-b636-0564339e9c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:22.654723 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6844256-a8f6-459b-b636-0564339e9c9b None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:22.655652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6844256-a8f6-459b-b636-0564339e9c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:22.656113 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6844256-a8f6-459b-b636-0564339e9c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:22.657523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6844256-a8f6-459b-b636-0564339e9c9b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:22.658689 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a451a-3662-4cca-a8bc-6bbe34df9a6b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:22.673745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 355/722] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:22.675246 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584f7e7-6111-4a13-883c-527c60e16080 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:22.675526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584f7e7-6111-4a13-883c-527c60e16080 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:22.675855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584f7e7-6111-4a13-883c-527c60e16080 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:22.681526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 356/723] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:22.681526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584f7e7-6111-4a13-883c-527c60e16080 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:22.701198 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236aba5-6bef-4094-89b8-a8707e12f4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:22.702145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236aba5-6bef-4094-89b8-a8707e12f4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:22.702689 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236aba5-6bef-4094-89b8-a8707e12f4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:22.703178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236aba5-6bef-4094-89b8-a8707e12f4c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:22.711430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236aba5-6bef-4094-89b8-a8707e12f4c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:22.713317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236aba5-6bef-4094-89b8-a8707e12f4c1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-9236aba5-6bef-4094-89b8-a8707e12f4c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:22.726357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:22.727516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:22.728580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:22.728580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:22.728580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:22.728580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:22.728580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:22.728580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:22.728580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:22.728580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:22.728580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:05:22.728580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:22.736313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 368/724] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Sep 23 15:05:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:22.738001 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e1312-1f6d-4635-ac61-d24bdf57a38d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:22.738490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e1312-1f6d-4635-ac61-d24bdf57a38d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:22.738946 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e1312-1f6d-4635-ac61-d24bdf57a38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:22.759599 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6d9e1312-1f6d-4635-ac61-d24bdf57a38d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:22.761273 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6d9e1312-1f6d-4635-ac61-d24bdf57a38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:22.791739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584f7e7-6111-4a13-883c-527c60e16080 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:22.831231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008e145-0229-4d7c-af7d-a570589d7feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:22.831895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008e145-0229-4d7c-af7d-a570589d7feb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:22.832173 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008e145-0229-4d7c-af7d-a570589d7feb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:22.832439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008e145-0229-4d7c-af7d-a570589d7feb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:22.844807 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008e145-0229-4d7c-af7d-a570589d7feb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:22.844807 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008e145-0229-4d7c-af7d-a570589d7feb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:22.845074 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 369/725] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:22.845683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e1312-1f6d-4635-ac61-d24bdf57a38d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-e008e145-0229-4d7c-af7d-a570589d7feb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:22.860345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:22.861404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:22.862340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:22.862340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:22.862340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:22.862340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:22.862340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:22.862340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:22.862340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:22.862340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:22.862340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 357/726] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:22.867916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008e145-0229-4d7c-af7d-a570589d7feb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:22.884982 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:22.980308 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008e145-0229-4d7c-af7d-a570589d7feb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:23.016057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:23.039746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e67028-0dfa-45d2-9607-db6ec0cf9192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:23.040407 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e67028-0dfa-45d2-9607-db6ec0cf9192 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:23.040706 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e67028-0dfa-45d2-9607-db6ec0cf9192 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:23.044253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e67028-0dfa-45d2-9607-db6ec0cf9192 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:23.045375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e67028-0dfa-45d2-9607-db6ec0cf9192 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:23.045688 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e67028-0dfa-45d2-9607-db6ec0cf9192 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:23.047787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e67028-0dfa-45d2-9607-db6ec0cf9192 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:23.063365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 358/727] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:23.064820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b145f67-056e-4c03-86d9-4cacee321d07 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:23.065098 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b145f67-056e-4c03-86d9-4cacee321d07 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:23.065368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b145f67-056e-4c03-86d9-4cacee321d07 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:23.066894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 359/728] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:23.068698 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b145f67-056e-4c03-86d9-4cacee321d07 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:23.082584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e1312-1f6d-4635-ac61-d24bdf57a38d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:23.121490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de343a-e771-4676-b154-21271791e55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:23.122470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de343a-e771-4676-b154-21271791e55a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:23.122679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de343a-e771-4676-b154-21271791e55a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:23.122943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de343a-e771-4676-b154-21271791e55a None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:23.123715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de343a-e771-4676-b154-21271791e55a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:23.124020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de343a-e771-4676-b154-21271791e55a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:23.125470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de343a-e771-4676-b154-21271791e55a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:23.148490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 370/729] 10.222.0.75 () {56 vars in 1149 bytes} [Fri Sep 23 15:05:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:23.148490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667f414-7db6-465d-b088-a77bc32d766b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:23.148490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667f414-7db6-465d-b088-a77bc32d766b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:23.148490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667f414-7db6-465d-b088-a77bc32d766b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:23.149534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:23.165052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e667f414-7db6-465d-b088-a77bc32d766b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:23.166350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e667f414-7db6-465d-b088-a77bc32d766b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:23.186661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b145f67-056e-4c03-86d9-4cacee321d07 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:23.222092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f491c-6f34-41e4-ad97-a373d4ffccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:23.222691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f491c-6f34-41e4-ad97-a373d4ffccdb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:23.222967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f491c-6f34-41e4-ad97-a373d4ffccdb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:23.223234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f491c-6f34-41e4-ad97-a373d4ffccdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:23.223957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f491c-6f34-41e4-ad97-a373d4ffccdb None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=2af1ba390fff41a68ff1a0035901c318) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:23.224257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f491c-6f34-41e4-ad97-a373d4ffccdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:23.225724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f491c-6f34-41e4-ad97-a373d4ffccdb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:23.231762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 360/730] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:23 2022] GET /identity/v3/role_assignments?user.id=2af1ba390fff41a68ff1a0035901c318&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=ae444a489623434999d758b71ae2dc87 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:23.232519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f491c-6f34-41e4-ad97-a373d4ffccdb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:23.257402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 371/731] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:23.257402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667f414-7db6-465d-b088-a77bc32d766b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:23.350994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:23.440977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f491c-6f34-41e4-ad97-a373d4ffccdb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:23.458279 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667f414-7db6-465d-b088-a77bc32d766b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:23.488956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:23.495493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ae7fd-df61-4a75-9d55-615a80eb4a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:23.496415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ae7fd-df61-4a75-9d55-615a80eb4a50 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:23.496923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ae7fd-df61-4a75-9d55-615a80eb4a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:23.497199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ae7fd-df61-4a75-9d55-615a80eb4a50 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:23.497952 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ae7fd-df61-4a75-9d55-615a80eb4a50 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:23.499304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23763025-fe74-4b7b-a78d-219199c13d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:23.499432 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ae7fd-df61-4a75-9d55-615a80eb4a50 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:23.499916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23763025-fe74-4b7b-a78d-219199c13d14 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:23.500190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23763025-fe74-4b7b-a78d-219199c13d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:23.500453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23763025-fe74-4b7b-a78d-219199c13d14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:23.521684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 361/732] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:05:23 2022] POST /identity/v3/services => generated 251 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:05:23.541569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ae7fd-df61-4a75-9d55-615a80eb4a50 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:23.597049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23763025-fe74-4b7b-a78d-219199c13d14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:23.598581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23763025-fe74-4b7b-a78d-219199c13d14 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:23.663056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:23.777781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 372/733] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 15:05:23.783386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23763025-fe74-4b7b-a78d-219199c13d14 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:23.797254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ae7fd-df61-4a75-9d55-615a80eb4a50 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:23.837429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad30f3-7931-48f4-b9e2-f8f0105ab989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:23.838072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad30f3-7931-48f4-b9e2-f8f0105ab989 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:23.838342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad30f3-7931-48f4-b9e2-f8f0105ab989 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:23.838606 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad30f3-7931-48f4-b9e2-f8f0105ab989 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:23.850090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad30f3-7931-48f4-b9e2-f8f0105ab989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-adad30f3-7931-48f4-b9e2-f8f0105ab989 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:23.855906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:23.856908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:23.857746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:23.857746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:23.857746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:23.857746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:23.857746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:23.857746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:23.857746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:23.857746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:23.857746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:23.857746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:23.857746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:05:23.857746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:23.857746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 362/734] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:05:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:23.863234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad30f3-7931-48f4-b9e2-f8f0105ab989 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:23.913208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23763025-fe74-4b7b-a78d-219199c13d14 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:23.930903 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:23.972070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d36a8-b6b9-49e0-b239-e886099f02a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:23.972861 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d36a8-b6b9-49e0-b239-e886099f02a7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:23.973141 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d36a8-b6b9-49e0-b239-e886099f02a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:23.973411 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d36a8-b6b9-49e0-b239-e886099f02a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:23.988499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad30f3-7931-48f4-b9e2-f8f0105ab989 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:23.997821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d36a8-b6b9-49e0-b239-e886099f02a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:23.999417 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d36a8-b6b9-49e0-b239-e886099f02a7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:24.012511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:24.027023 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967cf270-09bd-46c9-ba8b-9dab563309c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:24.027655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967cf270-09bd-46c9-ba8b-9dab563309c8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:24.027927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967cf270-09bd-46c9-ba8b-9dab563309c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:24.028190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967cf270-09bd-46c9-ba8b-9dab563309c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-a98d36a8-b6b9-49e0-b239-e886099f02a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:24.028357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:24.030407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:24.032754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:24.032754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:24.032754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:24.033160 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967cf270-09bd-46c9-ba8b-9dab563309c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:24.033160 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967cf270-09bd-46c9-ba8b-9dab563309c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:24.033433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 373/735] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:24.033578 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967cf270-09bd-46c9-ba8b-9dab563309c8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:24.033716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d92bb-2294-4929-9055-1b29dc99050a None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:24.033716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d92bb-2294-4929-9055-1b29dc99050a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:24.033716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d92bb-2294-4929-9055-1b29dc99050a None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:24.033716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 374/736] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:24.037087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d92bb-2294-4929-9055-1b29dc99050a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:24.046981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 363/737] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:05:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:24.048553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f044ff-f1f8-4cc8-b056-bd3d62226830 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:24.049051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f044ff-f1f8-4cc8-b056-bd3d62226830 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:24.049526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f044ff-f1f8-4cc8-b056-bd3d62226830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:24.071514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-96f044ff-f1f8-4cc8-b056-bd3d62226830 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:24.073696 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-96f044ff-f1f8-4cc8-b056-bd3d62226830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:24.153478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 364/738] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 15:05:24.154809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f044ff-f1f8-4cc8-b056-bd3d62226830 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:24.190982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d92bb-2294-4929-9055-1b29dc99050a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:24.240968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:24.262153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d60f4-d6bc-4fd5-be24-5aba649b263d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:24.262844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d60f4-d6bc-4fd5-be24-5aba649b263d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:24.263024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d60f4-d6bc-4fd5-be24-5aba649b263d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:24.263291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d60f4-d6bc-4fd5-be24-5aba649b263d None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:24.263943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d60f4-d6bc-4fd5-be24-5aba649b263d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:24.264235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d60f4-d6bc-4fd5-be24-5aba649b263d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:24.265633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d60f4-d6bc-4fd5-be24-5aba649b263d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:24.294886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 375/739] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:24.300483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d60f4-d6bc-4fd5-be24-5aba649b263d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:24.325279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f044ff-f1f8-4cc8-b056-bd3d62226830 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:24.366876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dbbca-d03a-4fd0-8a3c-138ad454046a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:24.367506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dbbca-d03a-4fd0-8a3c-138ad454046a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:24.367774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dbbca-d03a-4fd0-8a3c-138ad454046a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:24.368113 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dbbca-d03a-4fd0-8a3c-138ad454046a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:24.384617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862dbbca-d03a-4fd0-8a3c-138ad454046a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:24.384617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862dbbca-d03a-4fd0-8a3c-138ad454046a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-862dbbca-d03a-4fd0-8a3c-138ad454046a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:24.395247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:24.396221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:24.397149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:24.397149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:24.397149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:24.397149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 365/740] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:24.401961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dbbca-d03a-4fd0-8a3c-138ad454046a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:24.453035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:24.505347 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:24.519325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dbbca-d03a-4fd0-8a3c-138ad454046a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:24.543513 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d60f4-d6bc-4fd5-be24-5aba649b263d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:24.560883 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd6b1a-4e49-4b69-8bf2-5b3555ecaabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:24.561747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd6b1a-4e49-4b69-8bf2-5b3555ecaabc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:24.562166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd6b1a-4e49-4b69-8bf2-5b3555ecaabc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:24.562570 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd6b1a-4e49-4b69-8bf2-5b3555ecaabc None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:24.563364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd6b1a-4e49-4b69-8bf2-5b3555ecaabc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:24.563849 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd6b1a-4e49-4b69-8bf2-5b3555ecaabc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:24.565412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd6b1a-4e49-4b69-8bf2-5b3555ecaabc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:24.578114 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 366/741] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:24.579311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd6b1a-4e49-4b69-8bf2-5b3555ecaabc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:24.585912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bb05b-e1d8-4223-99ff-a06fedc04975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:24.586718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bb05b-e1d8-4223-99ff-a06fedc04975 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:24.587040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bb05b-e1d8-4223-99ff-a06fedc04975 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:24.587372 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6bb05b-e1d8-4223-99ff-a06fedc04975 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:24.596343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bb05b-e1d8-4223-99ff-a06fedc04975 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:24.598037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6bb05b-e1d8-4223-99ff-a06fedc04975 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-ba6bb05b-e1d8-4223-99ff-a06fedc04975 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:24.611212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:24.612151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:24.613064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:24.613064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:24.613064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:24.613064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:24.613064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:24.613064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:24.613064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:24.613064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:24.615411 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 376/742] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:24.616451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bb05b-e1d8-4223-99ff-a06fedc04975 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:24.706031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd6b1a-4e49-4b69-8bf2-5b3555ecaabc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:24.738696 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6bb05b-e1d8-4223-99ff-a06fedc04975 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:24.747894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719c835-800f-4a4b-a548-6be44c6e3d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:24.748626 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719c835-800f-4a4b-a548-6be44c6e3d2c None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:24.748977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719c835-800f-4a4b-a548-6be44c6e3d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:24.749320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719c835-800f-4a4b-a548-6be44c6e3d2c None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:24.750054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719c835-800f-4a4b-a548-6be44c6e3d2c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:24.751470 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719c835-800f-4a4b-a548-6be44c6e3d2c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:24.761104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:24.772316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 367/743] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:05:24 2022] POST /identity/v3/services => generated 240 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:05:24.773349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719c835-800f-4a4b-a548-6be44c6e3d2c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:24.791162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454c612-3edf-4210-8b42-e9e5dcd90a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:24.792050 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454c612-3edf-4210-8b42-e9e5dcd90a02 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:24.792412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454c612-3edf-4210-8b42-e9e5dcd90a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:24.792786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454c612-3edf-4210-8b42-e9e5dcd90a02 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:24.794485 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:24.804551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454c612-3edf-4210-8b42-e9e5dcd90a02 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:24.806231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454c612-3edf-4210-8b42-e9e5dcd90a02 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-2454c612-3edf-4210-8b42-e9e5dcd90a02 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:24.822396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:24.823474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:24.824924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:24.824924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:24.824924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:24.824924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:24.824924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:24.824924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:24.824924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:24.824924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:24.824924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 15:05:24.824924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:24.824924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 377/744] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:05:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:24.830646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454c612-3edf-4210-8b42-e9e5dcd90a02 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:24.926200 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719c835-800f-4a4b-a548-6be44c6e3d2c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:24.963644 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454c612-3edf-4210-8b42-e9e5dcd90a02 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:24.965960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70a840-7016-400c-ab11-d0c53aadd63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:24.966703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70a840-7016-400c-ab11-d0c53aadd63c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:24.967103 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70a840-7016-400c-ab11-d0c53aadd63c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:24.967439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70a840-7016-400c-ab11-d0c53aadd63c None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:24.968219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70a840-7016-400c-ab11-d0c53aadd63c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:24.968611 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70a840-7016-400c-ab11-d0c53aadd63c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:24.970173 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70a840-7016-400c-ab11-d0c53aadd63c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:24.991203 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 368/745] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:24.997358 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70a840-7016-400c-ab11-d0c53aadd63c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:25.007439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759c337-6ba0-4ef2-aebb-28bdbe0c6d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:25.008327 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759c337-6ba0-4ef2-aebb-28bdbe0c6d2c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:25.008910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759c337-6ba0-4ef2-aebb-28bdbe0c6d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:25.009360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759c337-6ba0-4ef2-aebb-28bdbe0c6d2c None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:25.011130 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759c337-6ba0-4ef2-aebb-28bdbe0c6d2c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:25.011660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759c337-6ba0-4ef2-aebb-28bdbe0c6d2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:25.013799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759c337-6ba0-4ef2-aebb-28bdbe0c6d2c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:25.045040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 378/746] 10.222.0.75 () {56 vars in 1135 bytes} [Fri Sep 23 15:05:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:25.051975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759c337-6ba0-4ef2-aebb-28bdbe0c6d2c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:25.088998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:25.101354 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:25.136049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70a840-7016-400c-ab11-d0c53aadd63c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:25.175530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d28d62-91f1-492e-94b3-6c9723670347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:25.176288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d28d62-91f1-492e-94b3-6c9723670347 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:25.176557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d28d62-91f1-492e-94b3-6c9723670347 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:25.176865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d28d62-91f1-492e-94b3-6c9723670347 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:25.177639 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d28d62-91f1-492e-94b3-6c9723670347 None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=17ff1e357746447a86d47d19daa18cf5) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:25.177940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d28d62-91f1-492e-94b3-6c9723670347 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:25.179355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d28d62-91f1-492e-94b3-6c9723670347 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:25.186426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 369/747] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:24 2022] GET /identity/v3/role_assignments?user.id=17ff1e357746447a86d47d19daa18cf5&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:25.192823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759c337-6ba0-4ef2-aebb-28bdbe0c6d2c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:25.234579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e36bf12-0951-433c-855c-502a5de8f156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:25.235500 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36bf12-0951-433c-855c-502a5de8f156 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:25.236014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36bf12-0951-433c-855c-502a5de8f156 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:25.236471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36bf12-0951-433c-855c-502a5de8f156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:25.315606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e36bf12-0951-433c-855c-502a5de8f156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:25.317882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e36bf12-0951-433c-855c-502a5de8f156 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:25.350805 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:25.438642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 379/748] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:05:25.443931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d28d62-91f1-492e-94b3-6c9723670347 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:25.565086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d28d62-91f1-492e-94b3-6c9723670347 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:25.603841 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f250d4e-ac40-4677-a4d1-76ae4b69f26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:25.604504 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f250d4e-ac40-4677-a4d1-76ae4b69f26b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:25.604803 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f250d4e-ac40-4677-a4d1-76ae4b69f26b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:25.605068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f250d4e-ac40-4677-a4d1-76ae4b69f26b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:25.611751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:25.628298 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f250d4e-ac40-4677-a4d1-76ae4b69f26b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:25.629765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f250d4e-ac40-4677-a4d1-76ae4b69f26b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-2f250d4e-ac40-4677-a4d1-76ae4b69f26b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:25.646759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:25.647765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:25.648989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:25.648989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:25.648989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:25.648989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 370/749] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:25.660940 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e36bf12-0951-433c-855c-502a5de8f156 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:25.848109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e36bf12-0951-433c-855c-502a5de8f156 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:25.890675 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:25.895808 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb4d53-5773-4a0b-ace1-2d804d00e41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:25.896921 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb4d53-5773-4a0b-ace1-2d804d00e41d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:25.897469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb4d53-5773-4a0b-ace1-2d804d00e41d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:25.898118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb4d53-5773-4a0b-ace1-2d804d00e41d None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:25.898782 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb4d53-5773-4a0b-ace1-2d804d00e41d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:25.899438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb4d53-5773-4a0b-ace1-2d804d00e41d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:25.901563 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb4d53-5773-4a0b-ace1-2d804d00e41d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:25.918490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 380/750] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:25.923685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f250d4e-ac40-4677-a4d1-76ae4b69f26b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:25.980224 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43445e-3dfe-44fd-a15f-f3e77905a998 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:25.980922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43445e-3dfe-44fd-a15f-f3e77905a998 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:25.981432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43445e-3dfe-44fd-a15f-f3e77905a998 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:25.985971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 381/751] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:05:25.990205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc43f6-3fff-4801-a011-1706f2feb36a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:25.990728 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc43f6-3fff-4801-a011-1706f2feb36a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:25.991262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc43f6-3fff-4801-a011-1706f2feb36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:26.026122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-dbdc43f6-3fff-4801-a011-1706f2feb36a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:26.027829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dbdc43f6-3fff-4801-a011-1706f2feb36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:26.125847 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f250d4e-ac40-4677-a4d1-76ae4b69f26b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:26.153663 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 382/752] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:26.183490 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7776df-8ca8-4e8f-9f17-f8bdda48c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:26.184126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7776df-8ca8-4e8f-9f17-f8bdda48c68f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:26.184392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7776df-8ca8-4e8f-9f17-f8bdda48c68f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:26.184676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7776df-8ca8-4e8f-9f17-f8bdda48c68f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:26.193631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7776df-8ca8-4e8f-9f17-f8bdda48c68f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:26.195059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7776df-8ca8-4e8f-9f17-f8bdda48c68f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-6f7776df-8ca8-4e8f-9f17-f8bdda48c68f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:26.209645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:26.210582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:26.212423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:26.212423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:26.212423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:26.212423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:26.212423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:26.212423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:26.212423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:26.212423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:26.212423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 371/753] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:26.216915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc43f6-3fff-4801-a011-1706f2feb36a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:26.266510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef8621-00b7-47be-a6a4-b299e4ad67e7 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:26.266872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef8621-00b7-47be-a6a4-b299e4ad67e7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:26.267031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef8621-00b7-47be-a6a4-b299e4ad67e7 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:26.268429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 372/754] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:05:26.274512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3ce8-686d-42e4-b77b-a256cf06f37d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:26.275094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3ce8-686d-42e4-b77b-a256cf06f37d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:26.275590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb3ce8-686d-42e4-b77b-a256cf06f37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:26.309642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-bfdb3ce8-686d-42e4-b77b-a256cf06f37d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:26.311065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-bfdb3ce8-686d-42e4-b77b-a256cf06f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:26.354929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc43f6-3fff-4801-a011-1706f2feb36a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:26.387569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:26.396507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:26.423034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c3970-c70b-4dec-9ed1-47a2d42687d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:26.423646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c3970-c70b-4dec-9ed1-47a2d42687d3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:26.424398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 373/755] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:26.424579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c3970-c70b-4dec-9ed1-47a2d42687d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:26.424883 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c3970-c70b-4dec-9ed1-47a2d42687d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:26.425650 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c3970-c70b-4dec-9ed1-47a2d42687d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:26.425954 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c3970-c70b-4dec-9ed1-47a2d42687d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:26.427392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c3970-c70b-4dec-9ed1-47a2d42687d3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:26.439693 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 383/756] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:26.449885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb3ce8-686d-42e4-b77b-a256cf06f37d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:26.505473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c3970-c70b-4dec-9ed1-47a2d42687d3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:26.614740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb3ce8-686d-42e4-b77b-a256cf06f37d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:26.630018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c3970-c70b-4dec-9ed1-47a2d42687d3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:26.670901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3a797-c4d4-4d01-82dc-efadf0df820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:26.672020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3a797-c4d4-4d01-82dc-efadf0df820b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:26.672759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3a797-c4d4-4d01-82dc-efadf0df820b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:26.673223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3a797-c4d4-4d01-82dc-efadf0df820b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:26.690857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703913b-7b84-4430-9902-8ee4c9d43848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:26.691883 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703913b-7b84-4430-9902-8ee4c9d43848 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:26.692356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703913b-7b84-4430-9902-8ee4c9d43848 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:26.693131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703913b-7b84-4430-9902-8ee4c9d43848 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:26.693850 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3a797-c4d4-4d01-82dc-efadf0df820b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:26.694785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703913b-7b84-4430-9902-8ee4c9d43848 None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=a8614f90cebb4a4eaf7ab5512927fb3c) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:26.698942 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703913b-7b84-4430-9902-8ee4c9d43848 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:26.700637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703913b-7b84-4430-9902-8ee4c9d43848 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-14a3a797-c4d4-4d01-82dc-efadf0df820b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:26.706146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:26.707080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:26.707995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:26.707995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:26.707995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:26.707995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:26.707995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:26.707995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:26.707995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:26.707995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:26.707995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:26.707995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:26.707995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:05:26.707995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:26.711976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 384/757] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:05:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:26.715659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 374/758] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:26 2022] GET /identity/v3/role_assignments?user.id=a8614f90cebb4a4eaf7ab5512927fb3c&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=ae444a489623434999d758b71ae2dc87 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:26.721921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703913b-7b84-4430-9902-8ee4c9d43848 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:26.786330 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3a797-c4d4-4d01-82dc-efadf0df820b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:26.787809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:26.852739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703913b-7b84-4430-9902-8ee4c9d43848 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:26.872944 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:26.894122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0f0e2-7af3-4ea1-af7e-11d3dd0961ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:26.895128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0f0e2-7af3-4ea1-af7e-11d3dd0961ec None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:26.895599 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0f0e2-7af3-4ea1-af7e-11d3dd0961ec None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:26.896061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0f0e2-7af3-4ea1-af7e-11d3dd0961ec None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:26.896962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac0f0e2-7af3-4ea1-af7e-11d3dd0961ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:26.897477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac0f0e2-7af3-4ea1-af7e-11d3dd0961ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:26.899068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac0f0e2-7af3-4ea1-af7e-11d3dd0961ec None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:26.909585 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 375/759] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:05:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:26.915910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0f0e2-7af3-4ea1-af7e-11d3dd0961ec None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:27.012995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3a797-c4d4-4d01-82dc-efadf0df820b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:27.032433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0f0e2-7af3-4ea1-af7e-11d3dd0961ec None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:27.035373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:27.064355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3af23d-18be-49ac-8a73-038d84c2cbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:27.065531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3af23d-18be-49ac-8a73-038d84c2cbdf None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:27.065809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3af23d-18be-49ac-8a73-038d84c2cbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:27.066076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3af23d-18be-49ac-8a73-038d84c2cbdf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:27.066941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3af23d-18be-49ac-8a73-038d84c2cbdf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:27.068633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3af23d-18be-49ac-8a73-038d84c2cbdf None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:27.077531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5567f-9fce-48a6-8611-ba0a013a8e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:27.078506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5567f-9fce-48a6-8611-ba0a013a8e8f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:27.079034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5567f-9fce-48a6-8611-ba0a013a8e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:27.079509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5567f-9fce-48a6-8611-ba0a013a8e8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-9c3af23d-18be-49ac-8a73-038d84c2cbdf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:27.080136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:27.081448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:27.082296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 385/760] 10.222.0.75 () {56 vars in 1129 bytes} [Fri Sep 23 15:05:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:27.087091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3af23d-18be-49ac-8a73-038d84c2cbdf None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:27.103845 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5567f-9fce-48a6-8611-ba0a013a8e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:27.110580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5567f-9fce-48a6-8611-ba0a013a8e8f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-55f5567f-9fce-48a6-8611-ba0a013a8e8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:27.138540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:27.139988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:27.140933 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:27.140933 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:27.140933 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:27.146603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 376/761] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:27.154832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5567f-9fce-48a6-8611-ba0a013a8e8f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:27.288959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5567f-9fce-48a6-8611-ba0a013a8e8f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:27.328438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c03211-62a7-429f-9e80-ef3a735c58a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:27.329622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c03211-62a7-429f-9e80-ef3a735c58a6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:27.331048 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c03211-62a7-429f-9e80-ef3a735c58a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:27.331695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c03211-62a7-429f-9e80-ef3a735c58a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:27.333123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c03211-62a7-429f-9e80-ef3a735c58a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:27.333837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c03211-62a7-429f-9e80-ef3a735c58a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:27.335949 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c03211-62a7-429f-9e80-ef3a735c58a6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:27.339879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3af23d-18be-49ac-8a73-038d84c2cbdf None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:27.355624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:27.364062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 377/762] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:27.370585 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c03211-62a7-429f-9e80-ef3a735c58a6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:27.404803 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:27.413448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be4301-e634-4484-8af5-52da8dd1b937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:27.414066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be4301-e634-4484-8af5-52da8dd1b937 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:27.414479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be4301-e634-4484-8af5-52da8dd1b937 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:27.414703 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be4301-e634-4484-8af5-52da8dd1b937 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:27.415527 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be4301-e634-4484-8af5-52da8dd1b937 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:27.415868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be4301-e634-4484-8af5-52da8dd1b937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:27.417630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be4301-e634-4484-8af5-52da8dd1b937 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:27.431018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 386/763] 10.222.0.75 () {56 vars in 1124 bytes} [Fri Sep 23 15:05:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:27.436323 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be4301-e634-4484-8af5-52da8dd1b937 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:27.498538 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c03211-62a7-429f-9e80-ef3a735c58a6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:27.540038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3346e03-1502-4db2-80df-f5ba608488e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:27.541067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346e03-1502-4db2-80df-f5ba608488e3 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:27.541544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346e03-1502-4db2-80df-f5ba608488e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:27.542084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3346e03-1502-4db2-80df-f5ba608488e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:27.554779 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3346e03-1502-4db2-80df-f5ba608488e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:27.556898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3346e03-1502-4db2-80df-f5ba608488e3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-b3346e03-1502-4db2-80df-f5ba608488e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:27.571083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:27.572126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:27.573554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:27.573554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:27.573554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:27.573554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:27.573554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:27.573554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:27.573554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:27.573554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:27.573554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:05:27.573554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:27.576074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 378/764] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Sep 23 15:05:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:27.582225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3346e03-1502-4db2-80df-f5ba608488e3 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:27.676961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:27.698712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be4301-e634-4484-8af5-52da8dd1b937 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:27.710632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3346e03-1502-4db2-80df-f5ba608488e3 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:27.736038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae9c3e-9b6a-4926-acd6-7dbb25e37cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:27.737025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae9c3e-9b6a-4926-acd6-7dbb25e37cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:27.737437 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae9c3e-9b6a-4926-acd6-7dbb25e37cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:27.737711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae9c3e-9b6a-4926-acd6-7dbb25e37cf5 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:27.738373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae9c3e-9b6a-4926-acd6-7dbb25e37cf5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:27.738670 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae9c3e-9b6a-4926-acd6-7dbb25e37cf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:27.740038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae9c3e-9b6a-4926-acd6-7dbb25e37cf5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:27.745364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 387/765] 10.222.0.75 () {56 vars in 1124 bytes} [Fri Sep 23 15:05:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:27.746548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e1c1e-3655-4626-9186-7a46b9a02802 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:27.746822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e1c1e-3655-4626-9186-7a46b9a02802 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:27.747083 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e1c1e-3655-4626-9186-7a46b9a02802 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:27.748454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 388/766] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:27.753518 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ed63c-da60-4e66-8d43-14da5b306d29 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:27.753810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ed63c-da60-4e66-8d43-14da5b306d29 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:27.754076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ed63c-da60-4e66-8d43-14da5b306d29 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:27.754255 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4276e9-384f-4e20-a891-c0428af143c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:27.755190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4276e9-384f-4e20-a891-c0428af143c0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:27.755297 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 389/767] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:27.755675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4276e9-384f-4e20-a891-c0428af143c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:27.755975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ed63c-da60-4e66-8d43-14da5b306d29 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:27.756208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4276e9-384f-4e20-a891-c0428af143c0 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:27.757261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4276e9-384f-4e20-a891-c0428af143c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:27.757618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4276e9-384f-4e20-a891-c0428af143c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:27.759353 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4276e9-384f-4e20-a891-c0428af143c0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:27.782621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 379/768] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Sep 23 15:05:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:27.783892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b13a9-4472-41f6-8eee-9631a240b507 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:27.784168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b13a9-4472-41f6-8eee-9631a240b507 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:27.784434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b13a9-4472-41f6-8eee-9631a240b507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:27.800208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:27.807521 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b91b13a9-4472-41f6-8eee-9631a240b507 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:27.809254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b91b13a9-4472-41f6-8eee-9631a240b507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:27.874545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ed63c-da60-4e66-8d43-14da5b306d29 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:27.888475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 380/769] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:27.889974 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bddc3-e1c7-4c8c-87c5-8577030db07e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:27.890081 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bddc3-e1c7-4c8c-87c5-8577030db07e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:27.890295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bddc3-e1c7-4c8c-87c5-8577030db07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:27.909960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a8eef-71d9-42ff-9b18-e309ee2bb48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:27.910813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a8eef-71d9-42ff-9b18-e309ee2bb48c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:27.911224 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a8eef-71d9-42ff-9b18-e309ee2bb48c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:27.911635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a8eef-71d9-42ff-9b18-e309ee2bb48c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:27.912469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a8eef-71d9-42ff-9b18-e309ee2bb48c None admin] RBAC: Adding query filter params (interface=public, service_id=7a6f6f7b46c14c71a53d8000ded42901, region_id=RegionOne) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:27.912942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a8eef-71d9-42ff-9b18-e309ee2bb48c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:27.914457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a8eef-71d9-42ff-9b18-e309ee2bb48c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:27.917317 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e96bddc3-e1c7-4c8c-87c5-8577030db07e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:27.918977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e96bddc3-e1c7-4c8c-87c5-8577030db07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:27.923278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 390/770] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Sep 23 15:05:27 2022] GET /identity/v3/endpoints?service_id=7a6f6f7b46c14c71a53d8000ded42901&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:27.924285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a8eef-71d9-42ff-9b18-e309ee2bb48c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:27.991280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:28.002751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 381/771] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:28.003658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bddc3-e1c7-4c8c-87c5-8577030db07e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:28.036711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:28.040667 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a8eef-71d9-42ff-9b18-e309ee2bb48c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:28.103553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688c2cd-81ef-4300-81b2-e73debaca228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:28.105586 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688c2cd-81ef-4300-81b2-e73debaca228 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:28.105860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688c2cd-81ef-4300-81b2-e73debaca228 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:28.106128 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688c2cd-81ef-4300-81b2-e73debaca228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:28.129727 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688c2cd-81ef-4300-81b2-e73debaca228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:28.132550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688c2cd-81ef-4300-81b2-e73debaca228 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:28.140508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bddc3-e1c7-4c8c-87c5-8577030db07e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-5688c2cd-81ef-4300-81b2-e73debaca228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:28.159506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:28.160478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:28.161391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:28.161391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:28.161391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:28.161391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 391/772] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:28.168122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688c2cd-81ef-4300-81b2-e73debaca228 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:28.194065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714031e1-fdcd-466b-8256-97e955fe573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:28.194666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714031e1-fdcd-466b-8256-97e955fe573c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:28.194935 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714031e1-fdcd-466b-8256-97e955fe573c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:28.195197 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714031e1-fdcd-466b-8256-97e955fe573c None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:28.195802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714031e1-fdcd-466b-8256-97e955fe573c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:28.197333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714031e1-fdcd-466b-8256-97e955fe573c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:28.203774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 382/773] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Sep 23 15:05:28 2022] GET /identity/v3/services => generated 772 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:28.233718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714031e1-fdcd-466b-8256-97e955fe573c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:28.358458 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:28.359363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688c2cd-81ef-4300-81b2-e73debaca228 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:28.397147 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d0165-cdd2-4dac-b784-0e847c8083bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:28.397766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d0165-cdd2-4dac-b784-0e847c8083bc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:28.398041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d0165-cdd2-4dac-b784-0e847c8083bc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:28.398363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d0165-cdd2-4dac-b784-0e847c8083bc None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:28.398962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d0165-cdd2-4dac-b784-0e847c8083bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:28.399353 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d0165-cdd2-4dac-b784-0e847c8083bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:28.401302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d0165-cdd2-4dac-b784-0e847c8083bc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:28.407373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714031e1-fdcd-466b-8256-97e955fe573c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:28.414634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 392/774] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:28.421355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d0165-cdd2-4dac-b784-0e847c8083bc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:28.435009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:28.458183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525cf69-82a1-45be-b62d-63a7d98932e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:28.459044 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525cf69-82a1-45be-b62d-63a7d98932e9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:28.459512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525cf69-82a1-45be-b62d-63a7d98932e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:28.459685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525cf69-82a1-45be-b62d-63a7d98932e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:28.474906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525cf69-82a1-45be-b62d-63a7d98932e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-c525cf69-82a1-45be-b62d-63a7d98932e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:28.479983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:28.481379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:28.482245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:28.482245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:28.482245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:28.482245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:28.482245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:28.482245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:28.482245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:28.482245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:28.482245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:28.482245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:28.482245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:28.482245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:28.482245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 383/775] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:28.482932 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525cf69-82a1-45be-b62d-63a7d98932e9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:28.550266 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d0165-cdd2-4dac-b784-0e847c8083bc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:28.588700 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8969dda-4ad8-4a8b-88db-a70a5e3235a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:28.589716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8969dda-4ad8-4a8b-88db-a70a5e3235a5 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:28.590189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8969dda-4ad8-4a8b-88db-a70a5e3235a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:28.590684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8969dda-4ad8-4a8b-88db-a70a5e3235a5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:28.591467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8969dda-4ad8-4a8b-88db-a70a5e3235a5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:28.593147 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8969dda-4ad8-4a8b-88db-a70a5e3235a5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d8969dda-4ad8-4a8b-88db-a70a5e3235a5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:28.602709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:28.603966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 15:05:28.604821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:28.607545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 393/776] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:28 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:28.608937 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8969dda-4ad8-4a8b-88db-a70a5e3235a5 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:28.623349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525cf69-82a1-45be-b62d-63a7d98932e9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:28.645329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:28.666627 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1e98f-3163-4b46-b19c-3fb720af8eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:28.667267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1e98f-3163-4b46-b19c-3fb720af8eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:28.667540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1e98f-3163-4b46-b19c-3fb720af8eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:28.667808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1e98f-3163-4b46-b19c-3fb720af8eb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:28.676337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1e98f-3163-4b46-b19c-3fb720af8eb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:28.677998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1e98f-3163-4b46-b19c-3fb720af8eb6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-c1c1e98f-3163-4b46-b19c-3fb720af8eb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:28.692703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:28.693635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:28.694608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:28.694608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:28.694608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:28.694608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:28.694608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:28.694608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:28.694608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:28.694608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:28.694608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 384/777] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:28.695098 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1e98f-3163-4b46-b19c-3fb720af8eb6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:28.701899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:28.728789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8969dda-4ad8-4a8b-88db-a70a5e3235a5 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:28.766173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f0ea9-79d5-4ca3-ac5e-a2c82e76957a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:28.767154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f0ea9-79d5-4ca3-ac5e-a2c82e76957a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:28.767628 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f0ea9-79d5-4ca3-ac5e-a2c82e76957a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:28.768109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f0ea9-79d5-4ca3-ac5e-a2c82e76957a None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:28.769002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f0ea9-79d5-4ca3-ac5e-a2c82e76957a None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:28.769523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f0ea9-79d5-4ca3-ac5e-a2c82e76957a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:28.771417 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f0ea9-79d5-4ca3-ac5e-a2c82e76957a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:28.781002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 394/778] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:28.782053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f0ea9-79d5-4ca3-ac5e-a2c82e76957a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:28.813621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1e98f-3163-4b46-b19c-3fb720af8eb6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:28.853631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbf5a6-6079-4c1b-b341-75a6568eb64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:28.854572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbf5a6-6079-4c1b-b341-75a6568eb64f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:28.855028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbf5a6-6079-4c1b-b341-75a6568eb64f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:28.855473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dbf5a6-6079-4c1b-b341-75a6568eb64f None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:28.856281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbf5a6-6079-4c1b-b341-75a6568eb64f None admin] RBAC: Adding query filter params (name=network) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:28.856880 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbf5a6-6079-4c1b-b341-75a6568eb64f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:28.858854 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dbf5a6-6079-4c1b-b341-75a6568eb64f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:28.872334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 385/779] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Sep 23 15:05:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:28.872334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbf5a6-6079-4c1b-b341-75a6568eb64f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:28.918310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:28.923510 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f0ea9-79d5-4ca3-ac5e-a2c82e76957a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:28.968775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec310c-e70d-4735-abd5-c7b2da8de7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:28.969782 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec310c-e70d-4735-abd5-c7b2da8de7bb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:28.970329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec310c-e70d-4735-abd5-c7b2da8de7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:28.970741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec310c-e70d-4735-abd5-c7b2da8de7bb None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:28.972141 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec310c-e70d-4735-abd5-c7b2da8de7bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:28.972627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec310c-e70d-4735-abd5-c7b2da8de7bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:28.975304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec310c-e70d-4735-abd5-c7b2da8de7bb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:28.979580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:28.981025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dbf5a6-6079-4c1b-b341-75a6568eb64f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:28.995021 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 395/780] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:28.996700 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec310c-e70d-4735-abd5-c7b2da8de7bb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:29.021069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a7022-1079-48b8-b0f7-c71cce39465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:29.021873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a7022-1079-48b8-b0f7-c71cce39465b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:29.022243 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a7022-1079-48b8-b0f7-c71cce39465b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:29.022607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a7022-1079-48b8-b0f7-c71cce39465b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:29.034424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a7022-1079-48b8-b0f7-c71cce39465b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:29.036091 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a7022-1079-48b8-b0f7-c71cce39465b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-089a7022-1079-48b8-b0f7-c71cce39465b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:29.052914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:29.054139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:29.055235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:29.055235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:29.055235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:29.056894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 386/781] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:29.058002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a7022-1079-48b8-b0f7-c71cce39465b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:29.149042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec310c-e70d-4735-abd5-c7b2da8de7bb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:29.171052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:29.178984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a7022-1079-48b8-b0f7-c71cce39465b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:29.193811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae354d-303f-4510-846b-40470f0b71c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:29.194584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae354d-303f-4510-846b-40470f0b71c2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:29.194921 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae354d-303f-4510-846b-40470f0b71c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:29.195258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae354d-303f-4510-846b-40470f0b71c2 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:29.196076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae354d-303f-4510-846b-40470f0b71c2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:29.196385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae354d-303f-4510-846b-40470f0b71c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:29.197802 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae354d-303f-4510-846b-40470f0b71c2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:29.203081 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 396/782] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Sep 23 15:05:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:29.203993 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae354d-303f-4510-846b-40470f0b71c2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:29.226510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc805cb-983e-4952-91a7-76f329d199bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:29.227257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc805cb-983e-4952-91a7-76f329d199bf None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:29.228208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc805cb-983e-4952-91a7-76f329d199bf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:29.228503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc805cb-983e-4952-91a7-76f329d199bf None admin] PATH_INFO: `/v3/projects/27ac51280046479e82faaf3c0b550f72/users/2af1ba390fff41a68ff1a0035901c318/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:29.230501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc805cb-983e-4952-91a7-76f329d199bf None admin] RBAC: Authorizing `identity:create_grant(project_id=27ac51280046479e82faaf3c0b550f72, user_id=2af1ba390fff41a68ff1a0035901c318, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:29.231985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc805cb-983e-4952-91a7-76f329d199bf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:29.247734 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 387/783] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:05:29 2022] PUT /identity/v3/projects/27ac51280046479e82faaf3c0b550f72/users/2af1ba390fff41a68ff1a0035901c318/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:05:29.249087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238989c-41fb-411b-977c-afb45fadee5b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:29.249372 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238989c-41fb-411b-977c-afb45fadee5b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:29.249643 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238989c-41fb-411b-977c-afb45fadee5b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:29.252650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 388/784] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:29.252650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238989c-41fb-411b-977c-afb45fadee5b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:29.288426 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:29.352156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae354d-303f-4510-846b-40470f0b71c2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:29.392711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238989c-41fb-411b-977c-afb45fadee5b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:29.399825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05501a7-60f9-4835-b028-af825666df2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:29.400525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05501a7-60f9-4835-b028-af825666df2f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:29.400884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05501a7-60f9-4835-b028-af825666df2f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:29.401216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05501a7-60f9-4835-b028-af825666df2f None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:29.401993 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05501a7-60f9-4835-b028-af825666df2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:29.402352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05501a7-60f9-4835-b028-af825666df2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:29.404528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05501a7-60f9-4835-b028-af825666df2f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:29.422356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 397/785] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:29.424080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27974b-367f-411d-9653-396e746d79ba None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:29.424429 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27974b-367f-411d-9653-396e746d79ba None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:29.425037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27974b-367f-411d-9653-396e746d79ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:29.440373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc28fe-ca22-4f06-86af-888a944447ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:29.441494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc28fe-ca22-4f06-86af-888a944447ac None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:29.441959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc28fe-ca22-4f06-86af-888a944447ac None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:29.442429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc28fe-ca22-4f06-86af-888a944447ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:29.443334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc28fe-ca22-4f06-86af-888a944447ac None admin] RBAC: Adding query filter params (interface=public, service_id=0f55131d564549869414b1f011184147, region_id=RegionOne) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:29.443814 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc28fe-ca22-4f06-86af-888a944447ac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:29.445422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc28fe-ca22-4f06-86af-888a944447ac None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:29.447288 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ba27974b-367f-411d-9653-396e746d79ba None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:29.448703 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ba27974b-367f-411d-9653-396e746d79ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:29.455841 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 389/786] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Sep 23 15:05:29 2022] GET /identity/v3/endpoints?service_id=0f55131d564549869414b1f011184147&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:29.457679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc28fe-ca22-4f06-86af-888a944447ac None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:29.476937 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:29.538243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 398/787] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:29.539602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27974b-367f-411d-9653-396e746d79ba None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:29.567639 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc28fe-ca22-4f06-86af-888a944447ac None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:29.602802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ff467-253b-4e0b-92c1-879f5242a622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:29.603807 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ff467-253b-4e0b-92c1-879f5242a622 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:29.604276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ff467-253b-4e0b-92c1-879f5242a622 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:29.604787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ff467-253b-4e0b-92c1-879f5242a622 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:29.606202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:29.615748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ff467-253b-4e0b-92c1-879f5242a622 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:29.617619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ff467-253b-4e0b-92c1-879f5242a622 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-f93ff467-253b-4e0b-92c1-879f5242a622 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:29.630757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:29.631754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:29.633198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:29.633198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:29.633198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:29.633198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:29.633198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:29.633198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:29.633198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:29.633198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:29.633198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:05:29.633198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:29.635966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 390/788] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Sep 23 15:05:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:29.641172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ff467-253b-4e0b-92c1-879f5242a622 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:29.666354 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27974b-367f-411d-9653-396e746d79ba None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:29.714411 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e552b1a-3f0a-4497-bf7d-c0a645da4e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:29.715034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e552b1a-3f0a-4497-bf7d-c0a645da4e8b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:29.715310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e552b1a-3f0a-4497-bf7d-c0a645da4e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:29.715577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e552b1a-3f0a-4497-bf7d-c0a645da4e8b None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:29.716206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e552b1a-3f0a-4497-bf7d-c0a645da4e8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:29.717620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e552b1a-3f0a-4497-bf7d-c0a645da4e8b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:29.733973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 399/789] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Sep 23 15:05:29 2022] GET /identity/v3/services => generated 772 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:29.888361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e552b1a-3f0a-4497-bf7d-c0a645da4e8b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:29.910688 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:29.949039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ff467-253b-4e0b-92c1-879f5242a622 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:29.991726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d5b04-1224-441d-88d9-3cdf47ce4fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:29.992331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d5b04-1224-441d-88d9-3cdf47ce4fcd None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:29.992618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d5b04-1224-441d-88d9-3cdf47ce4fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:29.993049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d5b04-1224-441d-88d9-3cdf47ce4fcd None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:29.994156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d5b04-1224-441d-88d9-3cdf47ce4fcd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:29.994451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d5b04-1224-441d-88d9-3cdf47ce4fcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:29.996002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d5b04-1224-441d-88d9-3cdf47ce4fcd None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:30.009290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e552b1a-3f0a-4497-bf7d-c0a645da4e8b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:30.022863 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 391/790] 10.222.0.75 () {56 vars in 1149 bytes} [Fri Sep 23 15:05:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:30.024151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d5b04-1224-441d-88d9-3cdf47ce4fcd None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:30.031417 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:30.056934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aebdc8-cbf2-43d8-8c3d-c559b5629f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:30.057573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aebdc8-cbf2-43d8-8c3d-c559b5629f00 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:30.057846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aebdc8-cbf2-43d8-8c3d-c559b5629f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:30.058119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aebdc8-cbf2-43d8-8c3d-c559b5629f00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:30.069196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aebdc8-cbf2-43d8-8c3d-c559b5629f00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-f0aebdc8-cbf2-43d8-8c3d-c559b5629f00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:30.074349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:30.075292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:30.076111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:30.076111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:30.076111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:30.076111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:30.076111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:30.076111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:30.076111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:30.076111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:30.076111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:30.076111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:30.076111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:05:30.076111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:30.076111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 400/791] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:05:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:30.082859 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aebdc8-cbf2-43d8-8c3d-c559b5629f00 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:30.178852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d5b04-1224-441d-88d9-3cdf47ce4fcd None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:30.216438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149de6cc-6b29-4351-aecc-0e42369ebd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:30.225267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149de6cc-6b29-4351-aecc-0e42369ebd4a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:30.225545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149de6cc-6b29-4351-aecc-0e42369ebd4a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:30.225867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149de6cc-6b29-4351-aecc-0e42369ebd4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:30.229311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:30.243694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149de6cc-6b29-4351-aecc-0e42369ebd4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:30.249837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149de6cc-6b29-4351-aecc-0e42369ebd4a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:30.256648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aebdc8-cbf2-43d8-8c3d-c559b5629f00 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-149de6cc-6b29-4351-aecc-0e42369ebd4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:30.274641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:30.275745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:30.276666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:30.276666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:30.276666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:30.276666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 392/792] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:30.283078 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149de6cc-6b29-4351-aecc-0e42369ebd4a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:30.293633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71f121-72d9-40c6-88aa-028c4c67a525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:30.294249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71f121-72d9-40c6-88aa-028c4c67a525 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:30.294522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71f121-72d9-40c6-88aa-028c4c67a525 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:30.294790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71f121-72d9-40c6-88aa-028c4c67a525 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:30.295474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71f121-72d9-40c6-88aa-028c4c67a525 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:30.296009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71f121-72d9-40c6-88aa-028c4c67a525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:30.298372 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71f121-72d9-40c6-88aa-028c4c67a525 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:30.307962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 401/793] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:05:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:30.317360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71f121-72d9-40c6-88aa-028c4c67a525 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:30.400966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:30.460754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149de6cc-6b29-4351-aecc-0e42369ebd4a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:30.514825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71f121-72d9-40c6-88aa-028c4c67a525 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:30.521623 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50a817-2789-4d58-bb3c-c4f26ad72a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:30.522238 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50a817-2789-4d58-bb3c-c4f26ad72a76 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:30.522505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50a817-2789-4d58-bb3c-c4f26ad72a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:30.522769 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50a817-2789-4d58-bb3c-c4f26ad72a76 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:30.523420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50a817-2789-4d58-bb3c-c4f26ad72a76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:30.523712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50a817-2789-4d58-bb3c-c4f26ad72a76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:30.525304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50a817-2789-4d58-bb3c-c4f26ad72a76 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:30.556615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 393/794] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:30.557448 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50a817-2789-4d58-bb3c-c4f26ad72a76 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:30.586867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60134311-704e-481d-8111-722cb1a629d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:30.588326 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60134311-704e-481d-8111-722cb1a629d4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:30.588828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60134311-704e-481d-8111-722cb1a629d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:30.589297 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60134311-704e-481d-8111-722cb1a629d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:30.613655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60134311-704e-481d-8111-722cb1a629d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:30.615468 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60134311-704e-481d-8111-722cb1a629d4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:30.618114 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-60134311-704e-481d-8111-722cb1a629d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:30.645951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:30.646981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:30.647840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:30.647840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:30.647840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:30.655300 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 402/795] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:30.656297 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60134311-704e-481d-8111-722cb1a629d4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:30.713734 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50a817-2789-4d58-bb3c-c4f26ad72a76 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:30.780672 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ef50c-1285-4b69-879a-df2d7042939b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:30.781329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef50c-1285-4b69-879a-df2d7042939b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:30.781595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef50c-1285-4b69-879a-df2d7042939b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:30.781856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef50c-1285-4b69-879a-df2d7042939b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:30.788897 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ef50c-1285-4b69-879a-df2d7042939b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:30.790589 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ef50c-1285-4b69-879a-df2d7042939b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-420ef50c-1285-4b69-879a-df2d7042939b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:30.802528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:30.803731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:30.804980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:30.804980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:30.804980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:30.804980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:30.804980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:30.804980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:30.804980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:30.804980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:30.804980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 394/796] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:30.805625 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545df12-b3d1-4ebf-9a4a-fda24c5fe580 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:30.805625 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545df12-b3d1-4ebf-9a4a-fda24c5fe580 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:30.805781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545df12-b3d1-4ebf-9a4a-fda24c5fe580 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:30.807163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 395/797] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:30.812049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545df12-b3d1-4ebf-9a4a-fda24c5fe580 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:30.813657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:30.827058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60134311-704e-481d-8111-722cb1a629d4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:30.865975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937803d-c6eb-4765-899f-518e3d6c81b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:30.866886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937803d-c6eb-4765-899f-518e3d6c81b6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:30.867329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937803d-c6eb-4765-899f-518e3d6c81b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:30.868056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937803d-c6eb-4765-899f-518e3d6c81b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:30.869098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937803d-c6eb-4765-899f-518e3d6c81b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:30.869585 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937803d-c6eb-4765-899f-518e3d6c81b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:30.871121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937803d-c6eb-4765-899f-518e3d6c81b6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:30.883905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 403/798] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:30.890471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98c441-5c0e-4336-bde4-f8e57ab752f5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:30.890973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98c441-5c0e-4336-bde4-f8e57ab752f5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:30.891405 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98c441-5c0e-4336-bde4-f8e57ab752f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:30.912104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6d98c441-5c0e-4336-bde4-f8e57ab752f5 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:30.913681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6d98c441-5c0e-4336-bde4-f8e57ab752f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:30.925602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545df12-b3d1-4ebf-9a4a-fda24c5fe580 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:30.936085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:30.962985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371c5a2-c471-4ebe-bfeb-77f2adc735a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:30.963585 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371c5a2-c471-4ebe-bfeb-77f2adc735a7 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:30.963853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371c5a2-c471-4ebe-bfeb-77f2adc735a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:30.964152 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371c5a2-c471-4ebe-bfeb-77f2adc735a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:30.964983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371c5a2-c471-4ebe-bfeb-77f2adc735a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:30.965280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371c5a2-c471-4ebe-bfeb-77f2adc735a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:30.966740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371c5a2-c471-4ebe-bfeb-77f2adc735a7 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:30.980624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 396/799] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:30.981594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371c5a2-c471-4ebe-bfeb-77f2adc735a7 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:30.990074 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 404/800] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:30.991093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98c441-5c0e-4336-bde4-f8e57ab752f5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:31.073373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:31.093568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98c441-5c0e-4336-bde4-f8e57ab752f5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:31.130869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a411852-81a6-4fef-bc07-1b380ed21064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:31.131887 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a411852-81a6-4fef-bc07-1b380ed21064 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:31.132330 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a411852-81a6-4fef-bc07-1b380ed21064 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:31.132803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a411852-81a6-4fef-bc07-1b380ed21064 None admin] PATH_INFO: `/v3/projects/27ac51280046479e82faaf3c0b550f72/users/17ff1e357746447a86d47d19daa18cf5/roles/351badda50da4e45ac1cb31482331c06` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:31.159012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a411852-81a6-4fef-bc07-1b380ed21064 None admin] RBAC: Authorizing `identity:create_grant(project_id=27ac51280046479e82faaf3c0b550f72, user_id=17ff1e357746447a86d47d19daa18cf5, role_id=351badda50da4e45ac1cb31482331c06)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:31.161484 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a411852-81a6-4fef-bc07-1b380ed21064 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:31.165785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371c5a2-c471-4ebe-bfeb-77f2adc735a7 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:31.191775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 405/801] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:05:30 2022] PUT /identity/v3/projects/27ac51280046479e82faaf3c0b550f72/users/17ff1e357746447a86d47d19daa18cf5/roles/351badda50da4e45ac1cb31482331c06 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:05:31.215189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76463060-db90-4d52-9163-4acf4443ffc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:31.216550 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76463060-db90-4d52-9163-4acf4443ffc1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:31.217040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76463060-db90-4d52-9163-4acf4443ffc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:31.217446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76463060-db90-4d52-9163-4acf4443ffc1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:31.226347 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76463060-db90-4d52-9163-4acf4443ffc1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:31.228384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76463060-db90-4d52-9163-4acf4443ffc1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-76463060-db90-4d52-9163-4acf4443ffc1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:31.242350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:31.243471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:31.244554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:31.244554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:31.244554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:31.244554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:31.244554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:31.244554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:31.244554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:31.244554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:31.244554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 15:05:31.244554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:31.246920 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 397/802] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:05:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:31.253184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a411852-81a6-4fef-bc07-1b380ed21064 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:31.376039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76463060-db90-4d52-9163-4acf4443ffc1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:31.385209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a411852-81a6-4fef-bc07-1b380ed21064 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:31.433995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:31.444097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:31.460973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402f20e-4983-4cd7-9da4-2138c9bda6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:31.461925 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402f20e-4983-4cd7-9da4-2138c9bda6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:31.462167 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402f20e-4983-4cd7-9da4-2138c9bda6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:31.462430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402f20e-4983-4cd7-9da4-2138c9bda6d1 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:31.463172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402f20e-4983-4cd7-9da4-2138c9bda6d1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:31.463476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402f20e-4983-4cd7-9da4-2138c9bda6d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:31.465624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402f20e-4983-4cd7-9da4-2138c9bda6d1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:31.488014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 406/803] 10.222.0.75 () {56 vars in 1135 bytes} [Fri Sep 23 15:05:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:31.495471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402f20e-4983-4cd7-9da4-2138c9bda6d1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:31.560310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76463060-db90-4d52-9163-4acf4443ffc1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:31.597879 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f05ae-fce6-42e0-940c-2cad66d8e876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:31.598563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f05ae-fce6-42e0-940c-2cad66d8e876 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:31.598837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f05ae-fce6-42e0-940c-2cad66d8e876 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:31.599105 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f05ae-fce6-42e0-940c-2cad66d8e876 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:31.599690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f05ae-fce6-42e0-940c-2cad66d8e876 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:31.601089 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f05ae-fce6-42e0-940c-2cad66d8e876 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-202f05ae-fce6-42e0-940c-2cad66d8e876 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:31.624646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:31.625873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:31.626758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 398/804] 10.222.0.75 () {56 vars in 1129 bytes} [Fri Sep 23 15:05:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:31.629659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f05ae-fce6-42e0-940c-2cad66d8e876 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:31.667391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402f20e-4983-4cd7-9da4-2138c9bda6d1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:31.746410 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c33d6-d5cd-45c4-80a9-8926df7a236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:31.747353 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c33d6-d5cd-45c4-80a9-8926df7a236d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:31.747804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c33d6-d5cd-45c4-80a9-8926df7a236d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:31.748210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c33d6-d5cd-45c4-80a9-8926df7a236d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:31.773720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c33d6-d5cd-45c4-80a9-8926df7a236d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:31.775324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c33d6-d5cd-45c4-80a9-8926df7a236d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:31.795566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-030c33d6-d5cd-45c4-80a9-8926df7a236d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:31.803810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:31.804986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:31.806912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:31.806912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:31.806912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:31.810865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 407/805] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:31.813904 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f05ae-fce6-42e0-940c-2cad66d8e876 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:31.816337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c33d6-d5cd-45c4-80a9-8926df7a236d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:31.885507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bd9c3-24d0-4682-adae-51fb8e32e4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:31.886060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bd9c3-24d0-4682-adae-51fb8e32e4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:31.886331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bd9c3-24d0-4682-adae-51fb8e32e4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:31.886600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bd9c3-24d0-4682-adae-51fb8e32e4a2 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:31.887259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bd9c3-24d0-4682-adae-51fb8e32e4a2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:31.887554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bd9c3-24d0-4682-adae-51fb8e32e4a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:31.888971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bd9c3-24d0-4682-adae-51fb8e32e4a2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:31.893552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:31.894643 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 399/806] 10.222.0.75 () {56 vars in 1124 bytes} [Fri Sep 23 15:05:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:31.901371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bd9c3-24d0-4682-adae-51fb8e32e4a2 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:32.015493 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c33d6-d5cd-45c4-80a9-8926df7a236d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:32.049801 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60808770-0d61-43ed-81a1-cd52feb91cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:32.050557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60808770-0d61-43ed-81a1-cd52feb91cce None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:32.050828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60808770-0d61-43ed-81a1-cd52feb91cce None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:32.051472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60808770-0d61-43ed-81a1-cd52feb91cce None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:32.052375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60808770-0d61-43ed-81a1-cd52feb91cce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:32.052746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60808770-0d61-43ed-81a1-cd52feb91cce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:32.054594 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60808770-0d61-43ed-81a1-cd52feb91cce None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:32.060740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bd9c3-24d0-4682-adae-51fb8e32e4a2 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:32.069199 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 408/807] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:32.073209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87870f7-922e-44be-9a10-406befa4befa None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:32.073633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87870f7-922e-44be-9a10-406befa4befa None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:32.074010 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87870f7-922e-44be-9a10-406befa4befa None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:32.075421 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 409/808] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:32.077849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87870f7-922e-44be-9a10-406befa4befa None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:32.109495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6109c-2977-47c4-9733-8971396a1052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:32.110110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6109c-2977-47c4-9733-8971396a1052 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:32.110374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6109c-2977-47c4-9733-8971396a1052 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:32.110642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6109c-2977-47c4-9733-8971396a1052 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:32.111991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6109c-2977-47c4-9733-8971396a1052 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:32.112717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6109c-2977-47c4-9733-8971396a1052 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:32.117656 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6109c-2977-47c4-9733-8971396a1052 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:32.128653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 400/809] 10.222.0.75 () {56 vars in 1124 bytes} [Fri Sep 23 15:05:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:32.130091 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60529e0c-17c2-4404-9472-a7b11aa9a536 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:32.130518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60529e0c-17c2-4404-9472-a7b11aa9a536 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:32.130951 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60529e0c-17c2-4404-9472-a7b11aa9a536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:32.158516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-60529e0c-17c2-4404-9472-a7b11aa9a536 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:32.160027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-60529e0c-17c2-4404-9472-a7b11aa9a536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:32.182736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:32.216928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:32.224164 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87870f7-922e-44be-9a10-406befa4befa None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:32.260866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 401/810] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:32.261596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095dd78-2e80-49e5-a739-b30e131fe030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:32.262339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095dd78-2e80-49e5-a739-b30e131fe030 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:32.262712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095dd78-2e80-49e5-a739-b30e131fe030 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:32.262886 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60529e0c-17c2-4404-9472-a7b11aa9a536 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:32.263168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095dd78-2e80-49e5-a739-b30e131fe030 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:32.270509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095dd78-2e80-49e5-a739-b30e131fe030 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:32.272069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095dd78-2e80-49e5-a739-b30e131fe030 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d095dd78-2e80-49e5-a739-b30e131fe030 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:32.283817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:32.285005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:32.286092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:32.286092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:32.286092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:32.286092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:32.286092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:32.286092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:32.286092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:32.286092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:32.287694 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 410/811] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:32.305865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095dd78-2e80-49e5-a739-b30e131fe030 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:32.391887 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60529e0c-17c2-4404-9472-a7b11aa9a536 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:32.429885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42c81f-877f-44cd-b53c-688a5acde3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:32.430645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c81f-877f-44cd-b53c-688a5acde3cd None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:32.431055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c81f-877f-44cd-b53c-688a5acde3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:32.431311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c81f-877f-44cd-b53c-688a5acde3cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:32.432097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42c81f-877f-44cd-b53c-688a5acde3cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:32.434108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42c81f-877f-44cd-b53c-688a5acde3cd None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:32.442914 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095dd78-2e80-49e5-a739-b30e131fe030 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:32.460766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 402/812] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:05:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:05:32.462410 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2c1a6-693c-4a89-962d-7ef68049dd96 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:32.462999 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2c1a6-693c-4a89-962d-7ef68049dd96 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:32.463392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2c1a6-693c-4a89-962d-7ef68049dd96 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:32.465366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 403/813] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:32.483555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f132432-89d6-4fc5-a4cf-aa88cfcbcadf None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:32.484298 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f132432-89d6-4fc5-a4cf-aa88cfcbcadf None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:32.484881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f132432-89d6-4fc5-a4cf-aa88cfcbcadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:32.505484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:32.517861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7f132432-89d6-4fc5-a4cf-aa88cfcbcadf None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:32.519216 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7f132432-89d6-4fc5-a4cf-aa88cfcbcadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:32.521071 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155aea73-d3fb-498a-b785-c294b7ae26ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:32.522383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155aea73-d3fb-498a-b785-c294b7ae26ef None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:32.522856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155aea73-d3fb-498a-b785-c294b7ae26ef None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:32.523326 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155aea73-d3fb-498a-b785-c294b7ae26ef None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:32.524392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155aea73-d3fb-498a-b785-c294b7ae26ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:32.525025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155aea73-d3fb-498a-b785-c294b7ae26ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:32.526749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155aea73-d3fb-498a-b785-c294b7ae26ef None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:32.555173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 411/814] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:32.556209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155aea73-d3fb-498a-b785-c294b7ae26ef None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:32.582537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:32.619980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 404/815] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:32.621361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f132432-89d6-4fc5-a4cf-aa88cfcbcadf None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:32.731796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155aea73-d3fb-498a-b785-c294b7ae26ef None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:32.744338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f132432-89d6-4fc5-a4cf-aa88cfcbcadf None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:32.779007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204890f-fed1-4d80-b8f5-0e77d5f688a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:32.779646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204890f-fed1-4d80-b8f5-0e77d5f688a0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:32.779916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204890f-fed1-4d80-b8f5-0e77d5f688a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:32.780181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204890f-fed1-4d80-b8f5-0e77d5f688a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:32.784099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63337a6-91c2-4af0-8abf-71254fc988bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:32.785051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63337a6-91c2-4af0-8abf-71254fc988bd None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:32.785524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63337a6-91c2-4af0-8abf-71254fc988bd None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:32.785988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63337a6-91c2-4af0-8abf-71254fc988bd None admin] PATH_INFO: `/v3/projects/27ac51280046479e82faaf3c0b550f72/users/a8614f90cebb4a4eaf7ab5512927fb3c/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:32.788102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63337a6-91c2-4af0-8abf-71254fc988bd None admin] RBAC: Authorizing `identity:create_grant(project_id=27ac51280046479e82faaf3c0b550f72, user_id=a8614f90cebb4a4eaf7ab5512927fb3c, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:32.789845 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63337a6-91c2-4af0-8abf-71254fc988bd None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:32.793398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:32.794229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204890f-fed1-4d80-b8f5-0e77d5f688a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7204890f-fed1-4d80-b8f5-0e77d5f688a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:32.799404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:32.800355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:32.801796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:32.801796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:32.801796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:32.801796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:32.801796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:32.801796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:32.801796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:32.801796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:32.801796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:32.801796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:32.801796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:05:32.801796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:32.805337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 412/816] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:05:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:32.809452 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204890f-fed1-4d80-b8f5-0e77d5f688a0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:32.815667 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 405/817] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:05:32 2022] PUT /identity/v3/projects/27ac51280046479e82faaf3c0b550f72/users/a8614f90cebb4a4eaf7ab5512927fb3c/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:05:32.899918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:32.918641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63337a6-91c2-4af0-8abf-71254fc988bd None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:32.935072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204890f-fed1-4d80-b8f5-0e77d5f688a0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:32.977675 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6232c5-9ddd-4979-81c5-69847c893312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:32.978342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6232c5-9ddd-4979-81c5-69847c893312 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:32.978639 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6232c5-9ddd-4979-81c5-69847c893312 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:32.979332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6232c5-9ddd-4979-81c5-69847c893312 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:32.979654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6232c5-9ddd-4979-81c5-69847c893312 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:32.979951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6232c5-9ddd-4979-81c5-69847c893312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:32.981450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6232c5-9ddd-4979-81c5-69847c893312 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:32.992573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 413/818] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:05:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:33.000633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6232c5-9ddd-4979-81c5-69847c893312 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:33.074226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63337a6-91c2-4af0-8abf-71254fc988bd None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:33.121719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae77ce9-8e5a-4397-93c0-b796df008d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:33.122369 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae77ce9-8e5a-4397-93c0-b796df008d9b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:33.122641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae77ce9-8e5a-4397-93c0-b796df008d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:33.122913 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae77ce9-8e5a-4397-93c0-b796df008d9b None admin] PATH_INFO: `/v3/services/network` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:33.123532 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae77ce9-8e5a-4397-93c0-b796df008d9b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:33.125019 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae77ce9-8e5a-4397-93c0-b796df008d9b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-eae77ce9-8e5a-4397-93c0-b796df008d9b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:33.135025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:33.136059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:33.136948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 406/819] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:32 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:33.144423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae77ce9-8e5a-4397-93c0-b796df008d9b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:33.259885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:33.314570 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae77ce9-8e5a-4397-93c0-b796df008d9b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:33.316036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6232c5-9ddd-4979-81c5-69847c893312 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:33.353056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dce4c-427d-4095-8c49-42027efa0759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:33.354196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dce4c-427d-4095-8c49-42027efa0759 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:33.354663 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dce4c-427d-4095-8c49-42027efa0759 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:33.355120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85dce4c-427d-4095-8c49-42027efa0759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:33.363526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:33.368796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85dce4c-427d-4095-8c49-42027efa0759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:33.370434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85dce4c-427d-4095-8c49-42027efa0759 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-a85dce4c-427d-4095-8c49-42027efa0759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:33.387189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:33.388180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:33.389324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:33.389324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:33.389324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:33.389324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 414/820] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:33.389910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc1dbc-3d67-41af-babf-2418cabeb07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:33.390532 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc1dbc-3d67-41af-babf-2418cabeb07a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:33.390802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc1dbc-3d67-41af-babf-2418cabeb07a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:33.391073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc1dbc-3d67-41af-babf-2418cabeb07a None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:33.391696 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc1dbc-3d67-41af-babf-2418cabeb07a None admin] RBAC: Adding query filter params (name=network) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:33.391990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc1dbc-3d67-41af-babf-2418cabeb07a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:33.396979 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc1dbc-3d67-41af-babf-2418cabeb07a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:33.398254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dce4c-427d-4095-8c49-42027efa0759 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:33.398505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 407/821] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Sep 23 15:05:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:33.404067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc1dbc-3d67-41af-babf-2418cabeb07a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:33.536134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc1dbc-3d67-41af-babf-2418cabeb07a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:33.563290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:33.585319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4be70f-6298-4938-b517-9831d6111849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:33.585943 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4be70f-6298-4938-b517-9831d6111849 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:33.586079 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85dce4c-427d-4095-8c49-42027efa0759 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:33.586597 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4be70f-6298-4938-b517-9831d6111849 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:33.586597 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4be70f-6298-4938-b517-9831d6111849 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:33.587152 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4be70f-6298-4938-b517-9831d6111849 None admin] RBAC: Adding query filter params (type=network) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:33.587481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4be70f-6298-4938-b517-9831d6111849 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:33.588929 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4be70f-6298-4938-b517-9831d6111849 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:33.601054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 408/822] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Sep 23 15:05:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:33.607233 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4be70f-6298-4938-b517-9831d6111849 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:33.624965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012d87b-b503-402c-99cc-c50adb97ad90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:33.625598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012d87b-b503-402c-99cc-c50adb97ad90 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:33.625869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012d87b-b503-402c-99cc-c50adb97ad90 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:33.626130 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d012d87b-b503-402c-99cc-c50adb97ad90 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:33.629180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012d87b-b503-402c-99cc-c50adb97ad90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:33.629545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012d87b-b503-402c-99cc-c50adb97ad90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:33.631047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d012d87b-b503-402c-99cc-c50adb97ad90 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:33.645985 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 415/823] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:33.657935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012d87b-b503-402c-99cc-c50adb97ad90 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:33.738971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4be70f-6298-4938-b517-9831d6111849 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:33.759161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:33.776242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9af6b9-b1fc-4d86-9ae1-56b9485291f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:33.776901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9af6b9-b1fc-4d86-9ae1-56b9485291f3 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:33.777176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9af6b9-b1fc-4d86-9ae1-56b9485291f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:33.777443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9af6b9-b1fc-4d86-9ae1-56b9485291f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:33.778165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9af6b9-b1fc-4d86-9ae1-56b9485291f3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:33.779561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9af6b9-b1fc-4d86-9ae1-56b9485291f3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:33.803054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 409/824] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:05:33 2022] POST /identity/v3/endpoints => generated 337 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:05:33.809897 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b463914-4a06-4322-845f-e24a0a1913e1 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:33.810545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b463914-4a06-4322-845f-e24a0a1913e1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:33.811013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b463914-4a06-4322-845f-e24a0a1913e1 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:33.814617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 410/825] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:33.819175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf06892-5d16-4ef3-bbea-3dcc11d9ec56 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:33.819761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf06892-5d16-4ef3-bbea-3dcc11d9ec56 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:33.820256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf06892-5d16-4ef3-bbea-3dcc11d9ec56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:33.836201 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d012d87b-b503-402c-99cc-c50adb97ad90 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:33.860120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8cf06892-5d16-4ef3-bbea-3dcc11d9ec56 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:33.862486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8cf06892-5d16-4ef3-bbea-3dcc11d9ec56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:33.916059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:33.936226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb983cf-4cb8-463d-ac24-f2ecca3ae9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:33.937301 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb983cf-4cb8-463d-ac24-f2ecca3ae9bd None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:33.939471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb983cf-4cb8-463d-ac24-f2ecca3ae9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:33.940048 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb983cf-4cb8-463d-ac24-f2ecca3ae9bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:33.957840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb983cf-4cb8-463d-ac24-f2ecca3ae9bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:33.959744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb983cf-4cb8-463d-ac24-f2ecca3ae9bd None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:33.971819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 411/826] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-efb983cf-4cb8-463d-ac24-f2ecca3ae9bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:33.980109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:33.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:33.982122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:33.982122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:33.982122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:33.982122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:33.982122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:33.982122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:33.982122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:33.982122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:33.982122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:05:33.982122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:33.988719 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 416/827] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Sep 23 15:05:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:33.991552 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf06892-5d16-4ef3-bbea-3dcc11d9ec56 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:34.117270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf06892-5d16-4ef3-bbea-3dcc11d9ec56 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:34.162794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd117b2-0560-4fe3-8def-eadb164ea812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:34.163773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd117b2-0560-4fe3-8def-eadb164ea812 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:34.164244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd117b2-0560-4fe3-8def-eadb164ea812 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:34.164927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd117b2-0560-4fe3-8def-eadb164ea812 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:34.166074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd117b2-0560-4fe3-8def-eadb164ea812 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:34.166584 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd117b2-0560-4fe3-8def-eadb164ea812 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:34.168198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd117b2-0560-4fe3-8def-eadb164ea812 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:34.191129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 412/828] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Sep 23 15:05:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:34.192813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb983cf-4cb8-463d-ac24-f2ecca3ae9bd None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:34.301258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb983cf-4cb8-463d-ac24-f2ecca3ae9bd None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:34.309234 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:34.338965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0a2bc-52f7-4dd5-bd92-2afbc70f05e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:34.339636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0a2bc-52f7-4dd5-bd92-2afbc70f05e0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:34.339915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0a2bc-52f7-4dd5-bd92-2afbc70f05e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:34.340239 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0a2bc-52f7-4dd5-bd92-2afbc70f05e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:34.350171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd117b2-0560-4fe3-8def-eadb164ea812 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:34.388905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:34.448562 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0a2bc-52f7-4dd5-bd92-2afbc70f05e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:34.450288 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0a2bc-52f7-4dd5-bd92-2afbc70f05e0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:34.467686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd117b2-0560-4fe3-8def-eadb164ea812 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:34.505583 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a2313-18d6-4adb-bce2-54d61abc826a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:34.506733 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a2313-18d6-4adb-bce2-54d61abc826a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:34.507256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a2313-18d6-4adb-bce2-54d61abc826a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:34.507740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a2313-18d6-4adb-bce2-54d61abc826a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:34.519312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a2313-18d6-4adb-bce2-54d61abc826a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-5e0a2313-18d6-4adb-bce2-54d61abc826a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:34.525111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:34.526026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:34.526853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:34.526853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:34.526853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:34.526853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:34.526853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:34.526853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:34.526853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:34.526853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:34.526853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:34.526853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:34.526853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:05:34.526853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:34.532760 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 413/829] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:05:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:34.538415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a2313-18d6-4adb-bce2-54d61abc826a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:34.580681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 417/830] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:34 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:05:34.585392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0a2bc-52f7-4dd5-bd92-2afbc70f05e0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:34.588488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:34.656903 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:34.668626 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a2313-18d6-4adb-bce2-54d61abc826a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:34.693469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0a2bc-52f7-4dd5-bd92-2afbc70f05e0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:34.710818 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926b932-45f8-4f31-a513-6be17e869402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:34.711426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926b932-45f8-4f31-a513-6be17e869402 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:34.711836 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926b932-45f8-4f31-a513-6be17e869402 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:34.712105 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926b932-45f8-4f31-a513-6be17e869402 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:34.712967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926b932-45f8-4f31-a513-6be17e869402 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:34.713298 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926b932-45f8-4f31-a513-6be17e869402 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:34.714642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926b932-45f8-4f31-a513-6be17e869402 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:34.730535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 414/831] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:05:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:34.731637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926b932-45f8-4f31-a513-6be17e869402 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:34.738579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f02c1-c35e-431e-b87e-178fbf75ae47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:34.739515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f02c1-c35e-431e-b87e-178fbf75ae47 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:34.739987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f02c1-c35e-431e-b87e-178fbf75ae47 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:34.740465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f02c1-c35e-431e-b87e-178fbf75ae47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:34.752135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f02c1-c35e-431e-b87e-178fbf75ae47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:34.754299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f02c1-c35e-431e-b87e-178fbf75ae47 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-f29f02c1-c35e-431e-b87e-178fbf75ae47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:34.770569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:34.771535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:34.772491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:34.772491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:34.772491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:34.774843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 418/832] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:34.780434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f02c1-c35e-431e-b87e-178fbf75ae47 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:34.841612 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:34.855660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926b932-45f8-4f31-a513-6be17e869402 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:34.891389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f02c1-c35e-431e-b87e-178fbf75ae47 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:34.897164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fd47e-d971-449e-999b-9cb5bb848758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:34.897804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fd47e-d971-449e-999b-9cb5bb848758 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:34.898078 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fd47e-d971-449e-999b-9cb5bb848758 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:34.898348 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030fd47e-d971-449e-999b-9cb5bb848758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:34.912151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030fd47e-d971-449e-999b-9cb5bb848758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:34.914099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030fd47e-d971-449e-999b-9cb5bb848758 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-030fd47e-d971-449e-999b-9cb5bb848758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:34.930752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:34.931804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:34.932768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:34.932768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:34.932768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:34.938610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 415/833] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:34.938769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dc0d3-e536-4bce-8f5e-eb624ba37d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:34.938769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dc0d3-e536-4bce-8f5e-eb624ba37d4d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:34.938769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dc0d3-e536-4bce-8f5e-eb624ba37d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:34.938769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dc0d3-e536-4bce-8f5e-eb624ba37d4d None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:34.939080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0dc0d3-e536-4bce-8f5e-eb624ba37d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:34.939080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0dc0d3-e536-4bce-8f5e-eb624ba37d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:34.939080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0dc0d3-e536-4bce-8f5e-eb624ba37d4d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:34.939679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fd47e-d971-449e-999b-9cb5bb848758 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:34.950224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:34.960188 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 419/834] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:34.961739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dc0d3-e536-4bce-8f5e-eb624ba37d4d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:35.059675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030fd47e-d971-449e-999b-9cb5bb848758 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:35.095310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dc0d3-e536-4bce-8f5e-eb624ba37d4d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:35.101423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803c02a-58c1-438a-8403-68a3b084284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:35.102392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803c02a-58c1-438a-8403-68a3b084284d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:35.102866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803c02a-58c1-438a-8403-68a3b084284d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:35.103329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803c02a-58c1-438a-8403-68a3b084284d None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:35.104259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803c02a-58c1-438a-8403-68a3b084284d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:35.104954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803c02a-58c1-438a-8403-68a3b084284d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:35.106649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803c02a-58c1-438a-8403-68a3b084284d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:35.113902 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:35.128434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 416/835] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:35.129787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803c02a-58c1-438a-8403-68a3b084284d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:35.135149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee553e0-e664-459c-a11d-368f69e5203a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:35.135149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee553e0-e664-459c-a11d-368f69e5203a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:35.135435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee553e0-e664-459c-a11d-368f69e5203a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:35.135645 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee553e0-e664-459c-a11d-368f69e5203a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:35.143087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee553e0-e664-459c-a11d-368f69e5203a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:35.144266 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee553e0-e664-459c-a11d-368f69e5203a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-4ee553e0-e664-459c-a11d-368f69e5203a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:35.157207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:35.158156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:35.159037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:35.159037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:35.159037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:35.159037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:35.159037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:35.159037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:35.159037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:35.159037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:35.163171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 420/836] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:35.164358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee553e0-e664-459c-a11d-368f69e5203a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:35.206065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:35.244345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803c02a-58c1-438a-8403-68a3b084284d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:35.269587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee553e0-e664-459c-a11d-368f69e5203a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:35.281031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9efd8-d8f1-4d8a-bbcd-a59676661e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:35.281662 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9efd8-d8f1-4d8a-bbcd-a59676661e05 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:35.281934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9efd8-d8f1-4d8a-bbcd-a59676661e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:35.282208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9efd8-d8f1-4d8a-bbcd-a59676661e05 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:35.290361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9efd8-d8f1-4d8a-bbcd-a59676661e05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:35.291508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9efd8-d8f1-4d8a-bbcd-a59676661e05 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:35.304068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132cc49-a4b4-4b6b-b05b-5ea49cbd7d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:35.304787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132cc49-a4b4-4b6b-b05b-5ea49cbd7d65 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:35.305059 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132cc49-a4b4-4b6b-b05b-5ea49cbd7d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:35.305385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132cc49-a4b4-4b6b-b05b-5ea49cbd7d65 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-17d9efd8-d8f1-4d8a-bbcd-a59676661e05 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:35.305549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:35.306427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:35.312637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:35.312637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:35.312637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:35.312637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:35.312637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:35.312637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:35.312637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:35.312637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:35.312637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:05:35.312637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:35.313202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132cc49-a4b4-4b6b-b05b-5ea49cbd7d65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:35.313279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 417/837] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Sep 23 15:05:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:35.313359 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132cc49-a4b4-4b6b-b05b-5ea49cbd7d65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:35.313429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd98f-b315-4dbe-a3dd-0a00e8bd3a03 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:35.313429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd98f-b315-4dbe-a3dd-0a00e8bd3a03 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:35.313550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132cc49-a4b4-4b6b-b05b-5ea49cbd7d65 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:35.313617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd98f-b315-4dbe-a3dd-0a00e8bd3a03 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:35.314657 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 418/838] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:35.315531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dd98f-b315-4dbe-a3dd-0a00e8bd3a03 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:35.332349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 421/839] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:35.332349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f4bb1-b81e-4a7d-b3c6-147f41c69889 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:35.332349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f4bb1-b81e-4a7d-b3c6-147f41c69889 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:35.332349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f4bb1-b81e-4a7d-b3c6-147f41c69889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:35.348556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f86f4bb1-b81e-4a7d-b3c6-147f41c69889 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:35.349748 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f86f4bb1-b81e-4a7d-b3c6-147f41c69889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:35.381233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:35.424648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 422/840] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:35.425649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f4bb1-b81e-4a7d-b3c6-147f41c69889 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:35.443203 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dd98f-b315-4dbe-a3dd-0a00e8bd3a03 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:35.470072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:35.479667 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84034798-e084-44a6-8dfe-cf4e805307c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:35.480265 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84034798-e084-44a6-8dfe-cf4e805307c2 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:35.480599 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84034798-e084-44a6-8dfe-cf4e805307c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:35.480832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84034798-e084-44a6-8dfe-cf4e805307c2 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:35.481555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84034798-e084-44a6-8dfe-cf4e805307c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:35.481857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84034798-e084-44a6-8dfe-cf4e805307c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:35.483609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84034798-e084-44a6-8dfe-cf4e805307c2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:35.506027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 419/841] 10.222.0.75 () {56 vars in 1149 bytes} [Fri Sep 23 15:05:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:35.506027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f53083-761f-4f7c-831e-fb3d084a76a9 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:35.506027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f53083-761f-4f7c-831e-fb3d084a76a9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:35.506027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f53083-761f-4f7c-831e-fb3d084a76a9 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:35.508643 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 420/842] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:35.509487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f53083-761f-4f7c-831e-fb3d084a76a9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:35.545512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f4bb1-b81e-4a7d-b3c6-147f41c69889 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:35.580051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3765884-927a-4148-bc43-89227de19fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:35.580671 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3765884-927a-4148-bc43-89227de19fdc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:35.580942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3765884-927a-4148-bc43-89227de19fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:35.581204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3765884-927a-4148-bc43-89227de19fdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:35.581947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3765884-927a-4148-bc43-89227de19fdc None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=2af1ba390fff41a68ff1a0035901c318) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:35.582248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3765884-927a-4148-bc43-89227de19fdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:35.583776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3765884-927a-4148-bc43-89227de19fdc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:35.589649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 423/843] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:35 2022] GET /identity/v3/role_assignments?user.id=2af1ba390fff41a68ff1a0035901c318&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=ae444a489623434999d758b71ae2dc87 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:35.590866 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a562d-e09d-4e28-968c-6dca9a7b0d15 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:35.591139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a562d-e09d-4e28-968c-6dca9a7b0d15 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:35.591403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a562d-e09d-4e28-968c-6dca9a7b0d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:35.612678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-762a562d-e09d-4e28-968c-6dca9a7b0d15 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:35.613881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-762a562d-e09d-4e28-968c-6dca9a7b0d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:35.631579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f53083-761f-4f7c-831e-fb3d084a76a9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:35.640956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:35.683305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b110a-9420-4d2b-a8c6-2b639ccd3ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:35.684156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b110a-9420-4d2b-a8c6-2b639ccd3ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:35.684627 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b110a-9420-4d2b-a8c6-2b639ccd3ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:35.685318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b110a-9420-4d2b-a8c6-2b639ccd3ec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:35.700367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 424/844] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:35.702365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a562d-e09d-4e28-968c-6dca9a7b0d15 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:35.765054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:35.777296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b110a-9420-4d2b-a8c6-2b639ccd3ec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:35.779302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b110a-9420-4d2b-a8c6-2b639ccd3ec8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:35.850343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a562d-e09d-4e28-968c-6dca9a7b0d15 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:35.900605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ada1d-7e26-4090-9108-d705eb376817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:35.901254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ada1d-7e26-4090-9108-d705eb376817 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:35.901521 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ada1d-7e26-4090-9108-d705eb376817 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:35.901780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ada1d-7e26-4090-9108-d705eb376817 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:35.902378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ada1d-7e26-4090-9108-d705eb376817 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:35.903716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ada1d-7e26-4090-9108-d705eb376817 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:35.919969 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 421/845] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:35 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d97ada1d-7e26-4090-9108-d705eb376817 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:35.920159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:35.921296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:35.922231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 425/846] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:05:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:35.925112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b110a-9420-4d2b-a8c6-2b639ccd3ec8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:35.926022 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ada1d-7e26-4090-9108-d705eb376817 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:35.953771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:36.017703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:36.037116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ada1d-7e26-4090-9108-d705eb376817 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:36.037346 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b110a-9420-4d2b-a8c6-2b639ccd3ec8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:36.076040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af1d0f-86a9-4d62-a4f2-a4fbeeac2e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:36.076673 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af1d0f-86a9-4d62-a4f2-a4fbeeac2e86 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:36.076945 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af1d0f-86a9-4d62-a4f2-a4fbeeac2e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:36.077206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af1d0f-86a9-4d62-a4f2-a4fbeeac2e86 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:36.077857 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af1d0f-86a9-4d62-a4f2-a4fbeeac2e86 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:36.078157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af1d0f-86a9-4d62-a4f2-a4fbeeac2e86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:36.079504 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af1d0f-86a9-4d62-a4f2-a4fbeeac2e86 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:36.080108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0baa9-8fad-4034-809d-912b9d802a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:36.080967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0baa9-8fad-4034-809d-912b9d802a88 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:36.081431 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0baa9-8fad-4034-809d-912b9d802a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:36.081884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0baa9-8fad-4034-809d-912b9d802a88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:36.084125 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 426/847] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:05:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:36.084921 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af1d0f-86a9-4d62-a4f2-a4fbeeac2e86 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:36.098576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0baa9-8fad-4034-809d-912b9d802a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:36.100127 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0baa9-8fad-4034-809d-912b9d802a88 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-90e0baa9-8fad-4034-809d-912b9d802a88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:36.120327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:36.121522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:36.122381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:36.122381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:36.122381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:36.126900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 422/848] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:36.127676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0baa9-8fad-4034-809d-912b9d802a88 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:36.190061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:36.207626 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af1d0f-86a9-4d62-a4f2-a4fbeeac2e86 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:36.241761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d0f09-bbc0-4dda-ab10-0912b51c840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:36.242382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d0f09-bbc0-4dda-ab10-0912b51c840e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:36.242650 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d0f09-bbc0-4dda-ab10-0912b51c840e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:36.243030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d0f09-bbc0-4dda-ab10-0912b51c840e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:36.251555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0baa9-8fad-4034-809d-912b9d802a88 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:36.256231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d0f09-bbc0-4dda-ab10-0912b51c840e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-709d0f09-bbc0-4dda-ab10-0912b51c840e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:36.261707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:36.262637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:36.263451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:36.263451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:36.263451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:36.263451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:36.263451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:36.263451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:36.263451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:36.263451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:36.263451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:36.263451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:36.263451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:05:36.263451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:36.263451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 427/849] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:05:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:36.264132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d0f09-bbc0-4dda-ab10-0912b51c840e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:36.295985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffbbea-398c-4e8c-86dc-a618d1d65d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:36.296616 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffbbea-398c-4e8c-86dc-a618d1d65d82 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:36.296906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffbbea-398c-4e8c-86dc-a618d1d65d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:36.297180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffbbea-398c-4e8c-86dc-a618d1d65d82 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:36.297819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffbbea-398c-4e8c-86dc-a618d1d65d82 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:36.298118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffbbea-398c-4e8c-86dc-a618d1d65d82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:36.299480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffbbea-398c-4e8c-86dc-a618d1d65d82 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:36.301938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:36.316966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 423/850] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:05:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:36.328751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffbbea-398c-4e8c-86dc-a618d1d65d82 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:36.396612 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d0f09-bbc0-4dda-ab10-0912b51c840e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:36.441635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743069c5-1c48-4fdf-9bf1-f33ba233b348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:36.442262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743069c5-1c48-4fdf-9bf1-f33ba233b348 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:36.442531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743069c5-1c48-4fdf-9bf1-f33ba233b348 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:36.442794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743069c5-1c48-4fdf-9bf1-f33ba233b348 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:36.443450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743069c5-1c48-4fdf-9bf1-f33ba233b348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:36.443745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743069c5-1c48-4fdf-9bf1-f33ba233b348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:36.445398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743069c5-1c48-4fdf-9bf1-f33ba233b348 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:36.461826 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 428/851] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:36.462133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffbbea-398c-4e8c-86dc-a618d1d65d82 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:36.469759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743069c5-1c48-4fdf-9bf1-f33ba233b348 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:36.482913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:36.505690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec39b02-0bd3-4cfd-bc86-4f482abd5868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:36.506553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec39b02-0bd3-4cfd-bc86-4f482abd5868 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:36.506993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec39b02-0bd3-4cfd-bc86-4f482abd5868 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:36.507524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec39b02-0bd3-4cfd-bc86-4f482abd5868 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:36.508519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec39b02-0bd3-4cfd-bc86-4f482abd5868 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:36.509020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec39b02-0bd3-4cfd-bc86-4f482abd5868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:36.510515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec39b02-0bd3-4cfd-bc86-4f482abd5868 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:36.520654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 424/852] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:05:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:36.526431 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec39b02-0bd3-4cfd-bc86-4f482abd5868 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:36.595337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743069c5-1c48-4fdf-9bf1-f33ba233b348 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:36.625650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:36.642241 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a9ad2-2c32-4394-9c91-5812105fbe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:36.643316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a9ad2-2c32-4394-9c91-5812105fbe1c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:36.644749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a9ad2-2c32-4394-9c91-5812105fbe1c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:36.645348 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a9ad2-2c32-4394-9c91-5812105fbe1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:36.656482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8a9ad2-2c32-4394-9c91-5812105fbe1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:36.659013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8a9ad2-2c32-4394-9c91-5812105fbe1c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:36.671205 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec39b02-0bd3-4cfd-bc86-4f482abd5868 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-ac8a9ad2-2c32-4394-9c91-5812105fbe1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:36.675322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:36.676183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:36.677115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:36.677115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:36.677115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:36.677115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:36.677115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:36.677115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:36.677115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:36.677115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:36.677115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 429/853] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:36.684248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a9ad2-2c32-4394-9c91-5812105fbe1c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:36.725083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6dd8b-7972-4d88-ad22-77145cb68510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:36.726634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6dd8b-7972-4d88-ad22-77145cb68510 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:36.727243 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6dd8b-7972-4d88-ad22-77145cb68510 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:36.727739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6dd8b-7972-4d88-ad22-77145cb68510 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:36.740581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6dd8b-7972-4d88-ad22-77145cb68510 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:36.742859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6dd8b-7972-4d88-ad22-77145cb68510 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-d2e6dd8b-7972-4d88-ad22-77145cb68510 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:36.762384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:36.763413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:36.764306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:36.764306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:36.764306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:36.766416 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 425/854] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:36.772153 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6dd8b-7972-4d88-ad22-77145cb68510 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:36.775134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:36.818914 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a9ad2-2c32-4394-9c91-5812105fbe1c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:36.862043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68489d-83fb-4ac1-93cd-3c40966939e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:36.865300 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68489d-83fb-4ac1-93cd-3c40966939e8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:36.865574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68489d-83fb-4ac1-93cd-3c40966939e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:36.865839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68489d-83fb-4ac1-93cd-3c40966939e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:36.866539 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68489d-83fb-4ac1-93cd-3c40966939e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:36.866835 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68489d-83fb-4ac1-93cd-3c40966939e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:36.868860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68489d-83fb-4ac1-93cd-3c40966939e8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:36.888791 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 430/855] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:36.891623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68489d-83fb-4ac1-93cd-3c40966939e8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:36.901323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6dd8b-7972-4d88-ad22-77145cb68510 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:36.920381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:36.951527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb05556-069b-479d-ba18-2aa2e9d93b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:36.952544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb05556-069b-479d-ba18-2aa2e9d93b74 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:36.953072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb05556-069b-479d-ba18-2aa2e9d93b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:36.953555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb05556-069b-479d-ba18-2aa2e9d93b74 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:36.954869 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb05556-069b-479d-ba18-2aa2e9d93b74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:36.955537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb05556-069b-479d-ba18-2aa2e9d93b74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:36.957132 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb05556-069b-479d-ba18-2aa2e9d93b74 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:36.978887 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 426/856] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:36.979276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb05556-069b-479d-ba18-2aa2e9d93b74 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:37.021604 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68489d-83fb-4ac1-93cd-3c40966939e8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:37.056677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:37.067601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaba546-b5d8-46da-ac95-f395f243d52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:37.068208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaba546-b5d8-46da-ac95-f395f243d52f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:37.068482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaba546-b5d8-46da-ac95-f395f243d52f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:37.068775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaba546-b5d8-46da-ac95-f395f243d52f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:37.069505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaba546-b5d8-46da-ac95-f395f243d52f None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=17ff1e357746447a86d47d19daa18cf5) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:37.069803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaba546-b5d8-46da-ac95-f395f243d52f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:37.071315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaba546-b5d8-46da-ac95-f395f243d52f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:37.084683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 431/857] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:36 2022] GET /identity/v3/role_assignments?user.id=17ff1e357746447a86d47d19daa18cf5&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:37.110762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb05556-069b-479d-ba18-2aa2e9d93b74 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:37.152304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27eee82-77ba-4e53-91f0-ca4f5e46db96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:37.153490 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27eee82-77ba-4e53-91f0-ca4f5e46db96 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:37.153977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27eee82-77ba-4e53-91f0-ca4f5e46db96 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:37.154442 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27eee82-77ba-4e53-91f0-ca4f5e46db96 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:37.163588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27eee82-77ba-4e53-91f0-ca4f5e46db96 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:37.166400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27eee82-77ba-4e53-91f0-ca4f5e46db96 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-e27eee82-77ba-4e53-91f0-ca4f5e46db96 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:37.179975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:37.187676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:37.189919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:37.189919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:37.189919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:37.189919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:37.189919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:37.189919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:37.189919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:37.189919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:37.189919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 15:05:37.189919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:37.193320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 427/858] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:05:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:37.200496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaba546-b5d8-46da-ac95-f395f243d52f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:37.360564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaba546-b5d8-46da-ac95-f395f243d52f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:37.420770 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b419c-fc48-4de7-a1f2-a99c0b241933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:37.422118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b419c-fc48-4de7-a1f2-a99c0b241933 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:37.422778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b419c-fc48-4de7-a1f2-a99c0b241933 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:37.423411 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b419c-fc48-4de7-a1f2-a99c0b241933 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:37.424768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b419c-fc48-4de7-a1f2-a99c0b241933 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:37.425482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b419c-fc48-4de7-a1f2-a99c0b241933 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:37.427916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b419c-fc48-4de7-a1f2-a99c0b241933 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:37.453787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 432/859] 10.222.0.75 () {56 vars in 1135 bytes} [Fri Sep 23 15:05:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:37.459975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27eee82-77ba-4e53-91f0-ca4f5e46db96 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:37.506572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:37.585517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27eee82-77ba-4e53-91f0-ca4f5e46db96 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:37.628802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f938b-d7a4-4ad8-8f3b-94d313b9c8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:37.629325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f938b-d7a4-4ad8-8f3b-94d313b9c8bc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:37.629596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f938b-d7a4-4ad8-8f3b-94d313b9c8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:37.629860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f938b-d7a4-4ad8-8f3b-94d313b9c8bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:37.710071 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f938b-d7a4-4ad8-8f3b-94d313b9c8bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:37.711613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f938b-d7a4-4ad8-8f3b-94d313b9c8bc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:37.749716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:37.827627 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 428/860] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:37 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:05:37.833135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b419c-fc48-4de7-a1f2-a99c0b241933 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:37.900525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:37.954650 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b419c-fc48-4de7-a1f2-a99c0b241933 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:37.992842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3d7b8-1681-43d2-900f-620b2ea294a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:37.993670 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3d7b8-1681-43d2-900f-620b2ea294a6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:37.993984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3d7b8-1681-43d2-900f-620b2ea294a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:37.994253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3d7b8-1681-43d2-900f-620b2ea294a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:38.005622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3d7b8-1681-43d2-900f-620b2ea294a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:38.006186 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484ec9f-9664-4f3a-a0a5-889cc150df7a None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:38.006470 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484ec9f-9664-4f3a-a0a5-889cc150df7a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:38.006745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484ec9f-9664-4f3a-a0a5-889cc150df7a None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:38.007081 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3d7b8-1681-43d2-900f-620b2ea294a6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:38.008420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 429/861] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:38.014047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642905a-3363-4e60-a735-970ebc7e4b52 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:38.014324 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642905a-3363-4e60-a735-970ebc7e4b52 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:38.014593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642905a-3363-4e60-a735-970ebc7e4b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-4cd3d7b8-1681-43d2-900f-620b2ea294a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:38.023339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:38.025097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:38.027286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:38.027286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:38.027286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:38.027286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 433/862] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:38.030968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3d7b8-1681-43d2-900f-620b2ea294a6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:38.037864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5642905a-3363-4e60-a735-970ebc7e4b52 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:38.039056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5642905a-3363-4e60-a735-970ebc7e4b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:38.121235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 430/863] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:38.156276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3d7b8-1681-43d2-900f-620b2ea294a6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:38.158208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:38.192292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6309035-5264-4248-bef8-53b17da8f264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:38.192931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6309035-5264-4248-bef8-53b17da8f264 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:38.193204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6309035-5264-4248-bef8-53b17da8f264 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:38.193469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6309035-5264-4248-bef8-53b17da8f264 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:38.194119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6309035-5264-4248-bef8-53b17da8f264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:38.194416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6309035-5264-4248-bef8-53b17da8f264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:38.195922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6309035-5264-4248-bef8-53b17da8f264 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:38.210048 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 434/864] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:38.216165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642905a-3363-4e60-a735-970ebc7e4b52 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:38.296217 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:38.330161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642905a-3363-4e60-a735-970ebc7e4b52 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:38.365990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422728b-cdde-429a-b991-7eb7411c43bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:38.366627 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422728b-cdde-429a-b991-7eb7411c43bb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:38.367181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422728b-cdde-429a-b991-7eb7411c43bb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:38.367447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422728b-cdde-429a-b991-7eb7411c43bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:38.374777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422728b-cdde-429a-b991-7eb7411c43bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:38.376223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422728b-cdde-429a-b991-7eb7411c43bb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-0422728b-cdde-429a-b991-7eb7411c43bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:38.388302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:38.389228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:38.390111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:38.390111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:38.390111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:38.390111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:38.390111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:38.390111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:38.390111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:38.390111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:38.390111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 431/865] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:38.395532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6309035-5264-4248-bef8-53b17da8f264 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:38.419842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422728b-cdde-429a-b991-7eb7411c43bb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:38.524271 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6309035-5264-4248-bef8-53b17da8f264 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:38.535374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422728b-cdde-429a-b991-7eb7411c43bb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:38.564743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:38.567595 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45452210-e502-4824-a529-e0bdb61c975e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:38.568216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45452210-e502-4824-a529-e0bdb61c975e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:38.568486 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45452210-e502-4824-a529-e0bdb61c975e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:38.568792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45452210-e502-4824-a529-e0bdb61c975e None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:38.569488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45452210-e502-4824-a529-e0bdb61c975e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:38.569787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45452210-e502-4824-a529-e0bdb61c975e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:38.572022 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45452210-e502-4824-a529-e0bdb61c975e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:38.581013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d91d16-f6da-4c48-9962-979b22e9ff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:38.582074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d91d16-f6da-4c48-9962-979b22e9ff49 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:38.582558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d91d16-f6da-4c48-9962-979b22e9ff49 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:38.583028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d91d16-f6da-4c48-9962-979b22e9ff49 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:38.583824 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d91d16-f6da-4c48-9962-979b22e9ff49 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:38.585520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d91d16-f6da-4c48-9962-979b22e9ff49 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:38.588660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 435/866] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:38.591488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45452210-e502-4824-a529-e0bdb61c975e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-c7d91d16-f6da-4c48-9962-979b22e9ff49 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:38.601274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:38.602434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 15:05:38.603530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:38.611800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 432/867] 10.222.0.75 () {56 vars in 1132 bytes} [Fri Sep 23 15:05:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:38.613118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d91d16-f6da-4c48-9962-979b22e9ff49 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:38.620415 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:38.700127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45452210-e502-4824-a529-e0bdb61c975e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:38.727944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d91d16-f6da-4c48-9962-979b22e9ff49 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:38.737234 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16244f-55d1-45e5-bca1-c859a33bc81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:38.737844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16244f-55d1-45e5-bca1-c859a33bc81a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:38.738115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16244f-55d1-45e5-bca1-c859a33bc81a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:38.738583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16244f-55d1-45e5-bca1-c859a33bc81a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:38.739091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16244f-55d1-45e5-bca1-c859a33bc81a None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=a8614f90cebb4a4eaf7ab5512927fb3c) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:38.739389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16244f-55d1-45e5-bca1-c859a33bc81a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:38.740842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16244f-55d1-45e5-bca1-c859a33bc81a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:38.747304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 436/868] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:38 2022] GET /identity/v3/role_assignments?user.id=a8614f90cebb4a4eaf7ab5512927fb3c&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=ae444a489623434999d758b71ae2dc87 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:38.750104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca21502-aad0-422b-9f15-ea20e06e0f27 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:38.750630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca21502-aad0-422b-9f15-ea20e06e0f27 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:38.751722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca21502-aad0-422b-9f15-ea20e06e0f27 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:38.758102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 437/869] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:38.760223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdd28b-ad16-4861-991a-da5c5a7f1fea None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:38.760512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdd28b-ad16-4861-991a-da5c5a7f1fea None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:38.760820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdd28b-ad16-4861-991a-da5c5a7f1fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:38.787277 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a3c14-bf4a-48c7-88a5-7c0bea5b4f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:38.788705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a3c14-bf4a-48c7-88a5-7c0bea5b4f63 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:38.789185 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a3c14-bf4a-48c7-88a5-7c0bea5b4f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:38.789838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a3c14-bf4a-48c7-88a5-7c0bea5b4f63 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:38.790766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a3c14-bf4a-48c7-88a5-7c0bea5b4f63 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:38.791305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a3c14-bf4a-48c7-88a5-7c0bea5b4f63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:38.792519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-26fdd28b-ad16-4861-991a-da5c5a7f1fea None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:38.793232 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a3c14-bf4a-48c7-88a5-7c0bea5b4f63 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:38.793823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-26fdd28b-ad16-4861-991a-da5c5a7f1fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:38.800507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 433/870] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Sep 23 15:05:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:38.806092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a3c14-bf4a-48c7-88a5-7c0bea5b4f63 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:38.884625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 438/871] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:38.894733 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:38.949720 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a3c14-bf4a-48c7-88a5-7c0bea5b4f63 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:38.987414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fc498-c503-4fa4-83d1-3688cdbec3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:38.987950 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fc498-c503-4fa4-83d1-3688cdbec3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:38.988222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fc498-c503-4fa4-83d1-3688cdbec3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:38.988487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fc498-c503-4fa4-83d1-3688cdbec3e4 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:38.989233 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933fc498-c503-4fa4-83d1-3688cdbec3e4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:38.989528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933fc498-c503-4fa4-83d1-3688cdbec3e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:38.990896 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933fc498-c503-4fa4-83d1-3688cdbec3e4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:38.996461 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 434/872] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Sep 23 15:05:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:39.162936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdd28b-ad16-4861-991a-da5c5a7f1fea None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:39.182886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:39.275709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdd28b-ad16-4861-991a-da5c5a7f1fea None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:39.313072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebd3e0-caca-48f2-a68e-9e1aa5cda986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:39.313681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebd3e0-caca-48f2-a68e-9e1aa5cda986 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:39.313954 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebd3e0-caca-48f2-a68e-9e1aa5cda986 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:39.314222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebd3e0-caca-48f2-a68e-9e1aa5cda986 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:39.314901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebd3e0-caca-48f2-a68e-9e1aa5cda986 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:39.316325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebd3e0-caca-48f2-a68e-9e1aa5cda986 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:39.334788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 439/873] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:05:39 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:05:39.527412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98163bd-e9bd-4711-8f25-ea86560c5f32 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:39.527603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98163bd-e9bd-4711-8f25-ea86560c5f32 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:39.527862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98163bd-e9bd-4711-8f25-ea86560c5f32 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:39.529501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 435/874] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:39.535818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441c0c9-e97e-4001-9feb-c419d43b0439 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:39.535818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441c0c9-e97e-4001-9feb-c419d43b0439 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:39.535818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441c0c9-e97e-4001-9feb-c419d43b0439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:39.559150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-8441c0c9-e97e-4001-9feb-c419d43b0439 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:39.560398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8441c0c9-e97e-4001-9feb-c419d43b0439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:39.637313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 440/875] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:39 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:39.914363 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98163bd-e9bd-4711-8f25-ea86560c5f32 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:40.019658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98163bd-e9bd-4711-8f25-ea86560c5f32 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:40.036466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:40.054905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef499083-d75b-4f5b-83f6-48cf625c0fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:40.055840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef499083-d75b-4f5b-83f6-48cf625c0fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:40.056244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef499083-d75b-4f5b-83f6-48cf625c0fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:40.056539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef499083-d75b-4f5b-83f6-48cf625c0fe0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:40.067609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef499083-d75b-4f5b-83f6-48cf625c0fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-ef499083-d75b-4f5b-83f6-48cf625c0fe0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:40.072989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:40.074076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:40.074992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:40.074992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:40.074992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:40.074992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:40.074992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:40.074992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:40.074992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:40.074992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:40.074992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:40.074992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:40.074992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:05:40.074992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:40.074992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 436/876] 10.222.0.75 () {56 vars in 1120 bytes} [Fri Sep 23 15:05:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:40.080180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441c0c9-e97e-4001-9feb-c419d43b0439 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:40.082416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:40.191506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441c0c9-e97e-4001-9feb-c419d43b0439 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:40.227168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a7826-9df0-4a76-a6b4-c318c7b1a964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:40.227772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a7826-9df0-4a76-a6b4-c318c7b1a964 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:40.228039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a7826-9df0-4a76-a6b4-c318c7b1a964 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:40.228441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a7826-9df0-4a76-a6b4-c318c7b1a964 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:40.229128 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a7826-9df0-4a76-a6b4-c318c7b1a964 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:40.229419 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a7826-9df0-4a76-a6b4-c318c7b1a964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:40.230844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a7826-9df0-4a76-a6b4-c318c7b1a964 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:40.243570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 441/877] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Sep 23 15:05:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:40.253695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef499083-d75b-4f5b-83f6-48cf625c0fe0 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:40.364486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef499083-d75b-4f5b-83f6-48cf625c0fe0 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:40.400035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e4fc6-74df-4668-8f0d-94572d3f7614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:40.400696 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e4fc6-74df-4668-8f0d-94572d3f7614 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:40.401058 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e4fc6-74df-4668-8f0d-94572d3f7614 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:40.401363 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e4fc6-74df-4668-8f0d-94572d3f7614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:40.411960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e4fc6-74df-4668-8f0d-94572d3f7614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:40.413440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e4fc6-74df-4668-8f0d-94572d3f7614 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-4c8e4fc6-74df-4668-8f0d-94572d3f7614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:40.429201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:40.430342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:40.432366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:40.432366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:40.432366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:40.432366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 437/878] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:40.442085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a7826-9df0-4a76-a6b4-c318c7b1a964 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:40.507450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:40.549795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a7826-9df0-4a76-a6b4-c318c7b1a964 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:40.584383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390d84e-233e-4253-b382-7b94970ea7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:40.585030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390d84e-233e-4253-b382-7b94970ea7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:40.585305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390d84e-233e-4253-b382-7b94970ea7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:40.585577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390d84e-233e-4253-b382-7b94970ea7c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:40.586231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390d84e-233e-4253-b382-7b94970ea7c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:40.586695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390d84e-233e-4253-b382-7b94970ea7c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:40.588167 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390d84e-233e-4253-b382-7b94970ea7c7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:40.601258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 442/879] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:40.606693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e4fc6-74df-4668-8f0d-94572d3f7614 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:40.637352 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:40.715017 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e4fc6-74df-4668-8f0d-94572d3f7614 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:40.750590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a056f9-c5db-4962-bf00-6b39331d44fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:40.751225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a056f9-c5db-4962-bf00-6b39331d44fc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:40.751500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a056f9-c5db-4962-bf00-6b39331d44fc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:40.751772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a056f9-c5db-4962-bf00-6b39331d44fc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:40.759999 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a056f9-c5db-4962-bf00-6b39331d44fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:40.761473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a056f9-c5db-4962-bf00-6b39331d44fc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-e3a056f9-c5db-4962-bf00-6b39331d44fc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:40.773337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:40.774367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:40.775593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:40.775593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:40.775593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:40.775593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:40.775593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:40.775593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:40.775593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:40.775593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:40.775593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:05:40.775593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:40.775593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 438/880] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Sep 23 15:05:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:40.782535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390d84e-233e-4253-b382-7b94970ea7c7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:40.889221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390d84e-233e-4253-b382-7b94970ea7c7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:40.917481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:40.931185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5c38f-8f47-4460-b52c-0667ae049a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:40.931844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5c38f-8f47-4460-b52c-0667ae049a2f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:40.932118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5c38f-8f47-4460-b52c-0667ae049a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:40.932382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5c38f-8f47-4460-b52c-0667ae049a2f None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:40.933160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5c38f-8f47-4460-b52c-0667ae049a2f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:40.933461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5c38f-8f47-4460-b52c-0667ae049a2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:40.934969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5c38f-8f47-4460-b52c-0667ae049a2f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:40.955034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 443/881] 10.222.0.75 () {56 vars in 1149 bytes} [Fri Sep 23 15:05:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:40.961074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a056f9-c5db-4962-bf00-6b39331d44fc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:41.030353 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ca1f1-3b15-432f-a1ba-ed274b026262 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:41.030551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ca1f1-3b15-432f-a1ba-ed274b026262 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:41.030802 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ca1f1-3b15-432f-a1ba-ed274b026262 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:41.032377 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 444/882] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:05:41.037318 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08562874-9884-4c56-bcc7-0c39cf3fd7e1 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:41.037591 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08562874-9884-4c56-bcc7-0c39cf3fd7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:41.037851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08562874-9884-4c56-bcc7-0c39cf3fd7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:41.055006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:41.057965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-08562874-9884-4c56-bcc7-0c39cf3fd7e1 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:41.059407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-08562874-9884-4c56-bcc7-0c39cf3fd7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:41.067259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a056f9-c5db-4962-bf00-6b39331d44fc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:41.103655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89a1f6-3ac0-4a0d-9426-312500f44457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:41.104270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89a1f6-3ac0-4a0d-9426-312500f44457 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:41.104545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89a1f6-3ac0-4a0d-9426-312500f44457 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:41.104844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89a1f6-3ac0-4a0d-9426-312500f44457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:41.156903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 445/883] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:41 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:41.197511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89a1f6-3ac0-4a0d-9426-312500f44457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:41.198964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89a1f6-3ac0-4a0d-9426-312500f44457 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:41.297296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:41.317045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 439/884] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:40 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:05:41.318816 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08562874-9884-4c56-bcc7-0c39cf3fd7e1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:41.393579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5853cae-feac-484c-a65e-48a1f9e5156b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:41.393948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5853cae-feac-484c-a65e-48a1f9e5156b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:41.394123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5853cae-feac-484c-a65e-48a1f9e5156b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:41.396846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 440/885] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:05:41.406419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f411456-ae5f-4345-905f-59b90c1c8291 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:41.406825 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f411456-ae5f-4345-905f-59b90c1c8291 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:41.407185 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f411456-ae5f-4345-905f-59b90c1c8291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:41.426748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0f411456-ae5f-4345-905f-59b90c1c8291 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:41.428004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0f411456-ae5f-4345-905f-59b90c1c8291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:41.432950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08562874-9884-4c56-bcc7-0c39cf3fd7e1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:41.447640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:41.471382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666c10f-a982-4a9d-990d-d87eb02e2aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:41.472012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666c10f-a982-4a9d-990d-d87eb02e2aac None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:41.472280 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666c10f-a982-4a9d-990d-d87eb02e2aac None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:41.472541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666c10f-a982-4a9d-990d-d87eb02e2aac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:41.484071 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666c10f-a982-4a9d-990d-d87eb02e2aac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:41.485849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666c10f-a982-4a9d-990d-d87eb02e2aac None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-5666c10f-a982-4a9d-990d-d87eb02e2aac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:41.501274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:41.502315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:41.503193 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:41.503193 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:41.503193 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:41.503193 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 446/886] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:41.503421 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666c10f-a982-4a9d-990d-d87eb02e2aac None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:41.508365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 441/887] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:41 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:41.509649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f411456-ae5f-4345-905f-59b90c1c8291 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:41.622484 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666c10f-a982-4a9d-990d-d87eb02e2aac None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:41.650404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f411456-ae5f-4345-905f-59b90c1c8291 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:41.660163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e57d99-499e-4701-920a-ff325b251580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:41.660976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e57d99-499e-4701-920a-ff325b251580 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:41.661174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e57d99-499e-4701-920a-ff325b251580 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:41.661402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e57d99-499e-4701-920a-ff325b251580 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:41.661999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e57d99-499e-4701-920a-ff325b251580 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:41.663361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e57d99-499e-4701-920a-ff325b251580 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-19e57d99-499e-4701-920a-ff325b251580 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:41.673072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:41.674148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:41.674972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:41.677033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:41.678484 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 447/888] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Sep 23 15:05:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:41.680936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e57d99-499e-4701-920a-ff325b251580 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:41.697395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047eb0b-28e6-4f25-8739-df31cded9f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:41.699044 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047eb0b-28e6-4f25-8739-df31cded9f2a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:41.699562 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047eb0b-28e6-4f25-8739-df31cded9f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:41.700317 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047eb0b-28e6-4f25-8739-df31cded9f2a None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:41.701244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047eb0b-28e6-4f25-8739-df31cded9f2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:41.701836 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047eb0b-28e6-4f25-8739-df31cded9f2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:41.703530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5047eb0b-28e6-4f25-8739-df31cded9f2a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:41.713475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:41.720684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 442/889] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:41.728641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047eb0b-28e6-4f25-8739-df31cded9f2a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:41.793222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e57d99-499e-4701-920a-ff325b251580 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:41.828399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde71af-f1d0-40b7-bfae-696b541f6d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:41.829199 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde71af-f1d0-40b7-bfae-696b541f6d7c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:41.829355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde71af-f1d0-40b7-bfae-696b541f6d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:41.829558 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde71af-f1d0-40b7-bfae-696b541f6d7c None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:41.830196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde71af-f1d0-40b7-bfae-696b541f6d7c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:41.830491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde71af-f1d0-40b7-bfae-696b541f6d7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:41.831870 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde71af-f1d0-40b7-bfae-696b541f6d7c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:41.845116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 448/890] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Sep 23 15:05:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:41.845116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9202b06-d617-418b-a941-0db398f6653b None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:41.845116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9202b06-d617-418b-a941-0db398f6653b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:41.845116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9202b06-d617-418b-a941-0db398f6653b None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:41.845116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 449/891] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:41.845116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9202b06-d617-418b-a941-0db398f6653b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:41.850711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047eb0b-28e6-4f25-8739-df31cded9f2a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:41.886769 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeadc8e-e142-4c3e-b6d3-6f16428a4b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:41.887310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeadc8e-e142-4c3e-b6d3-6f16428a4b70 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:41.887578 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeadc8e-e142-4c3e-b6d3-6f16428a4b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:41.887839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeadc8e-e142-4c3e-b6d3-6f16428a4b70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:41.894964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeadc8e-e142-4c3e-b6d3-6f16428a4b70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:41.896379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeadc8e-e142-4c3e-b6d3-6f16428a4b70 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-8eeadc8e-e142-4c3e-b6d3-6f16428a4b70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:41.907384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:41.908433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:41.909364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:41.909364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:41.909364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:41.909364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:41.909364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:41.909364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:41.909364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:41.909364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:41.909364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 443/892] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:41.913834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0af094-c82e-470d-ae6e-78ec41ad1b88 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:41.914113 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0af094-c82e-470d-ae6e-78ec41ad1b88 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:41.914389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0af094-c82e-470d-ae6e-78ec41ad1b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:41.933570 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5d0af094-c82e-470d-ae6e-78ec41ad1b88 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:41.934772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5d0af094-c82e-470d-ae6e-78ec41ad1b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:41.935275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:41.944951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9202b06-d617-418b-a941-0db398f6653b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:41.964490 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:41.980019 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88d9f9-fd5e-4422-95db-38f0915df532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:41.980658 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88d9f9-fd5e-4422-95db-38f0915df532 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:41.980933 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88d9f9-fd5e-4422-95db-38f0915df532 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:41.981200 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88d9f9-fd5e-4422-95db-38f0915df532 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:41.981861 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88d9f9-fd5e-4422-95db-38f0915df532 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:41.983228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88d9f9-fd5e-4422-95db-38f0915df532 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:42.000020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 450/893] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:05:41 2022] POST /identity/v3/services => generated 269 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:05:42.000859 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88d9f9-fd5e-4422-95db-38f0915df532 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:42.012221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 444/894] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:42.023525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0af094-c82e-470d-ae6e-78ec41ad1b88 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:42.138619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88d9f9-fd5e-4422-95db-38f0915df532 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:42.160888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0af094-c82e-470d-ae6e-78ec41ad1b88 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:42.179511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc44ad0-a0b5-402e-9023-004713f1ffb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:42.184916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc44ad0-a0b5-402e-9023-004713f1ffb3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:42.184916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc44ad0-a0b5-402e-9023-004713f1ffb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:42.184916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc44ad0-a0b5-402e-9023-004713f1ffb3 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:42.184916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc44ad0-a0b5-402e-9023-004713f1ffb3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:42.184916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc44ad0-a0b5-402e-9023-004713f1ffb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:42.184916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc44ad0-a0b5-402e-9023-004713f1ffb3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:42.191587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 451/895] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Sep 23 15:05:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:42.204175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e896e92-f556-4d3f-a349-b53ebdb41fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:42.204896 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e896e92-f556-4d3f-a349-b53ebdb41fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:42.205170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e896e92-f556-4d3f-a349-b53ebdb41fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:42.205436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e896e92-f556-4d3f-a349-b53ebdb41fb7 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:42.206150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e896e92-f556-4d3f-a349-b53ebdb41fb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:42.206451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e896e92-f556-4d3f-a349-b53ebdb41fb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:42.207912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e896e92-f556-4d3f-a349-b53ebdb41fb7 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:42.222491 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 445/896] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:42.228534 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc44ad0-a0b5-402e-9023-004713f1ffb3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:42.249746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:42.293692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e896e92-f556-4d3f-a349-b53ebdb41fb7 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:42.323096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:42.373381 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc44ad0-a0b5-402e-9023-004713f1ffb3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:42.419522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e896e92-f556-4d3f-a349-b53ebdb41fb7 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:42.423708 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625279e-916c-4954-90b5-d5af0b4dc852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:42.424458 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625279e-916c-4954-90b5-d5af0b4dc852 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:42.424973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625279e-916c-4954-90b5-d5af0b4dc852 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:42.425414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625279e-916c-4954-90b5-d5af0b4dc852 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:42.427242 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625279e-916c-4954-90b5-d5af0b4dc852 None admin] RBAC: Adding query filter params (role.id=ea4eebb7bada4fe4a89fa7f71f4b09ce, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=17ff1e357746447a86d47d19daa18cf5) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:42.427757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625279e-916c-4954-90b5-d5af0b4dc852 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:42.431797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a625279e-916c-4954-90b5-d5af0b4dc852 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:42.444731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 452/897] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:42 2022] GET /identity/v3/role_assignments?user.id=17ff1e357746447a86d47d19daa18cf5&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=ea4eebb7bada4fe4a89fa7f71f4b09ce => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:42.463786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9fb2c-d77e-45db-a5fb-231a987de05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:42.464447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9fb2c-d77e-45db-a5fb-231a987de05f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:42.464759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9fb2c-d77e-45db-a5fb-231a987de05f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:42.465100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9fb2c-d77e-45db-a5fb-231a987de05f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:42.465714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9fb2c-d77e-45db-a5fb-231a987de05f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:42.467134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9fb2c-d77e-45db-a5fb-231a987de05f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-00e9fb2c-d77e-45db-a5fb-231a987de05f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:42.477289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:42.478361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:42.479653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 446/898] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:05:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:42.485449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625279e-916c-4954-90b5-d5af0b4dc852 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:42.586162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:42.612201 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a625279e-916c-4954-90b5-d5af0b4dc852 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:42.657246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149add4e-43e2-4d07-a522-211e8020f383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:42.657886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149add4e-43e2-4d07-a522-211e8020f383 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:42.658155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149add4e-43e2-4d07-a522-211e8020f383 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:42.658417 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149add4e-43e2-4d07-a522-211e8020f383 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:42.659065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149add4e-43e2-4d07-a522-211e8020f383 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:42.659359 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149add4e-43e2-4d07-a522-211e8020f383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:42.660762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149add4e-43e2-4d07-a522-211e8020f383 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:42.666676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 453/899] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:05:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:42.672286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9fb2c-d77e-45db-a5fb-231a987de05f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:42.779170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:42.792210 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9fb2c-d77e-45db-a5fb-231a987de05f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:42.827914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d187c0-c8ca-41ad-a356-af5ea1fdbe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:42.829058 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d187c0-c8ca-41ad-a356-af5ea1fdbe21 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:42.829547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d187c0-c8ca-41ad-a356-af5ea1fdbe21 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:42.830038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d187c0-c8ca-41ad-a356-af5ea1fdbe21 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:42.830867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d187c0-c8ca-41ad-a356-af5ea1fdbe21 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:42.831363 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d187c0-c8ca-41ad-a356-af5ea1fdbe21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:42.832919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d187c0-c8ca-41ad-a356-af5ea1fdbe21 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:42.838752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 447/900] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:05:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:42.844830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149add4e-43e2-4d07-a522-211e8020f383 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:42.956843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149add4e-43e2-4d07-a522-211e8020f383 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:42.996978 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1849df-6056-4cee-8ead-535b73d1f93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:42.997963 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1849df-6056-4cee-8ead-535b73d1f93a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:42.998470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1849df-6056-4cee-8ead-535b73d1f93a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:42.998938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1849df-6056-4cee-8ead-535b73d1f93a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:42.999849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1849df-6056-4cee-8ead-535b73d1f93a None admin] RBAC: Adding query filter params (interface=public, service_id=bf1bdf694c7c4592826fe23b4b9b1779, region_id=RegionOne) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:43.000354 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1849df-6056-4cee-8ead-535b73d1f93a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:43.001933 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1849df-6056-4cee-8ead-535b73d1f93a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:43.008579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 454/901] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Sep 23 15:05:42 2022] GET /identity/v3/endpoints?service_id=bf1bdf694c7c4592826fe23b4b9b1779&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:43.016739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d187c0-c8ca-41ad-a356-af5ea1fdbe21 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:43.119043 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d187c0-c8ca-41ad-a356-af5ea1fdbe21 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:43.155512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e80d6c-5f82-4917-a4dd-bfec8b82152f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:43.156548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e80d6c-5f82-4917-a4dd-bfec8b82152f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:43.157036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e80d6c-5f82-4917-a4dd-bfec8b82152f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:43.157519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e80d6c-5f82-4917-a4dd-bfec8b82152f None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:43.158352 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e80d6c-5f82-4917-a4dd-bfec8b82152f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:43.159983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e80d6c-5f82-4917-a4dd-bfec8b82152f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:43.167386 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 448/902] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Sep 23 15:05:43 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:05:44.310119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0d2fb-560a-41d6-b2ca-3d4664a06c2f None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:44.310683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0d2fb-560a-41d6-b2ca-3d4664a06c2f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:44.310683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0d2fb-560a-41d6-b2ca-3d4664a06c2f None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:44.312278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 455/903] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:44.317825 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f9e5e-0991-439b-bd1a-6fb204bc7a70 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:44.318243 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f9e5e-0991-439b-bd1a-6fb204bc7a70 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:44.318424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558f9e5e-0991-439b-bd1a-6fb204bc7a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:44.337359 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-558f9e5e-0991-439b-bd1a-6fb204bc7a70 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:44.338563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-558f9e5e-0991-439b-bd1a-6fb204bc7a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:44.350983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:44.414327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 449/904] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:44.489580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebde46-deb3-40dd-a763-d6f54b669cd1 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:44.489858 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebde46-deb3-40dd-a763-d6f54b669cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:44.490130 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebde46-deb3-40dd-a763-d6f54b669cd1 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:44.491716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 456/905] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:44.497609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1926a0-2277-4bf4-a6a4-25efca250a06 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:44.498099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1926a0-2277-4bf4-a6a4-25efca250a06 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:44.498502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1926a0-2277-4bf4-a6a4-25efca250a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:44.517262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9d1926a0-2277-4bf4-a6a4-25efca250a06 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:44.518683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9d1926a0-2277-4bf4-a6a4-25efca250a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:44.592804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 450/906] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:44.684728 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebde46-deb3-40dd-a763-d6f54b669cd1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:44.698517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:44.808880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebde46-deb3-40dd-a763-d6f54b669cd1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:44.851682 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b4832-9790-40cb-947f-9784fd313d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:44.853076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b4832-9790-40cb-947f-9784fd313d12 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:44.853241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1926a0-2277-4bf4-a6a4-25efca250a06 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:44.853523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b4832-9790-40cb-947f-9784fd313d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:44.853955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b4832-9790-40cb-947f-9784fd313d12 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:44.854925 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b4832-9790-40cb-947f-9784fd313d12 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:44.857461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b4832-9790-40cb-947f-9784fd313d12 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-a20b4832-9790-40cb-947f-9784fd313d12 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:44.868373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:44.869416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:44.870316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 457/907] 10.222.0.75 () {56 vars in 1132 bytes} [Fri Sep 23 15:05:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:44.877169 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b4832-9790-40cb-947f-9784fd313d12 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:44.949834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:44.964512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:44.978750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1926a0-2277-4bf4-a6a4-25efca250a06 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:45.001719 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b4832-9790-40cb-947f-9784fd313d12 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:45.022800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73f2cd-b2cf-4263-abe5-f12ae150bc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:45.023750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73f2cd-b2cf-4263-abe5-f12ae150bc94 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:45.024226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73f2cd-b2cf-4263-abe5-f12ae150bc94 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:45.024822 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73f2cd-b2cf-4263-abe5-f12ae150bc94 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:45.025640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73f2cd-b2cf-4263-abe5-f12ae150bc94 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:45.027008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c73f2cd-b2cf-4263-abe5-f12ae150bc94 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:45.040808 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e489e-1765-4609-ab8f-fea0915706d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:45.041416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e489e-1765-4609-ab8f-fea0915706d0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:45.041683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e489e-1765-4609-ab8f-fea0915706d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:45.041953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e489e-1765-4609-ab8f-fea0915706d0 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:45.042567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e489e-1765-4609-ab8f-fea0915706d0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:45.042871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e489e-1765-4609-ab8f-fea0915706d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:45.051113 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 451/908] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:05:44 2022] POST /identity/v3/services => generated 241 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:05:45.051334 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e489e-1765-4609-ab8f-fea0915706d0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:45.051420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeffc4-a017-437e-bfaf-b578c5065772 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:45.051420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeffc4-a017-437e-bfaf-b578c5065772 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:45.051420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffeffc4-a017-437e-bfaf-b578c5065772 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:45.051420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 452/909] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:45.053239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf95664-bb29-4c1b-aa07-395e00f15cdd None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:45.053691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf95664-bb29-4c1b-aa07-395e00f15cdd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:45.054124 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf95664-bb29-4c1b-aa07-395e00f15cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:45.062483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 458/910] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Sep 23 15:05:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:45.063728 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e489e-1765-4609-ab8f-fea0915706d0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:45.080084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9bf95664-bb29-4c1b-aa07-395e00f15cdd None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:45.081434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9bf95664-bb29-4c1b-aa07-395e00f15cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:45.180835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 453/911] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:45.223031 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e489e-1765-4609-ab8f-fea0915706d0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:45.265762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b2404-1c64-41df-ae1e-2ff2df14a184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:45.266731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b2404-1c64-41df-ae1e-2ff2df14a184 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:45.267201 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b2404-1c64-41df-ae1e-2ff2df14a184 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:45.267464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b2404-1c64-41df-ae1e-2ff2df14a184 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:45.268118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b2404-1c64-41df-ae1e-2ff2df14a184 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:45.268441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b2404-1c64-41df-ae1e-2ff2df14a184 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:45.269868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b2404-1c64-41df-ae1e-2ff2df14a184 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:45.272429 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:45.280963 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 459/912] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Sep 23 15:05:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:45.284128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf95664-bb29-4c1b-aa07-395e00f15cdd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:45.349343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:45.387276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad48acd-5ff9-4dae-a78d-1840f80f2568 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:45.387829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad48acd-5ff9-4dae-a78d-1840f80f2568 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:45.388051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad48acd-5ff9-4dae-a78d-1840f80f2568 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:45.389812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 460/913] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:05:45.391945 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf95664-bb29-4c1b-aa07-395e00f15cdd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:45.395039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90239e72-2d09-4e55-85ae-ddef4577559d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:45.395450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90239e72-2d09-4e55-85ae-ddef4577559d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:45.395698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90239e72-2d09-4e55-85ae-ddef4577559d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:45.415119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-90239e72-2d09-4e55-85ae-ddef4577559d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:45.416482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-90239e72-2d09-4e55-85ae-ddef4577559d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:45.429171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842a303-c36a-466e-a11b-a642e8a0ea81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:45.429791 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842a303-c36a-466e-a11b-a642e8a0ea81 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:45.430092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842a303-c36a-466e-a11b-a642e8a0ea81 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:45.430406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842a303-c36a-466e-a11b-a642e8a0ea81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:45.431404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842a303-c36a-466e-a11b-a642e8a0ea81 None admin] RBAC: Adding query filter params (interface=public, service_id=ef156f3232cb4f3a870b0bb01f83c9fc, region_id=RegionOne) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:45.432211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842a303-c36a-466e-a11b-a642e8a0ea81 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:45.434180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842a303-c36a-466e-a11b-a642e8a0ea81 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:45.441669 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 454/914] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Sep 23 15:05:45 2022] GET /identity/v3/endpoints?service_id=ef156f3232cb4f3a870b0bb01f83c9fc&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:45.446821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842a303-c36a-466e-a11b-a642e8a0ea81 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:45.498439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 461/915] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:45.499716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90239e72-2d09-4e55-85ae-ddef4577559d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:45.567229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842a303-c36a-466e-a11b-a642e8a0ea81 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:45.601001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:45.612917 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b98588-62f1-4769-b778-acb6c141e32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:45.613528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b98588-62f1-4769-b778-acb6c141e32a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:45.613799 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b98588-62f1-4769-b778-acb6c141e32a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:45.614058 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b98588-62f1-4769-b778-acb6c141e32a None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:45.614664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b98588-62f1-4769-b778-acb6c141e32a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:45.616025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b98588-62f1-4769-b778-acb6c141e32a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:45.624690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 455/916] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Sep 23 15:05:45 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:05:45.653828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90239e72-2d09-4e55-85ae-ddef4577559d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:45.687795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:45.696521 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd27b3-1395-4c5d-b114-1024edb9e0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:45.697588 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd27b3-1395-4c5d-b114-1024edb9e0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:45.698059 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd27b3-1395-4c5d-b114-1024edb9e0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:45.698556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd27b3-1395-4c5d-b114-1024edb9e0d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:45.710797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd27b3-1395-4c5d-b114-1024edb9e0d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-97bd27b3-1395-4c5d-b114-1024edb9e0d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:45.716479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:45.717535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:45.718358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:45.718358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:45.718358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:45.718358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:45.718358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:45.718358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:45.718358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:45.718358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:45.718358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:45.718358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:45.718358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:05:45.718358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:45.721227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 462/917] 10.222.0.75 () {56 vars in 1120 bytes} [Fri Sep 23 15:05:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:45.727383 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b98588-62f1-4769-b778-acb6c141e32a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:45.802265 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd27b3-1395-4c5d-b114-1024edb9e0d3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:45.851020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b98588-62f1-4769-b778-acb6c141e32a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:45.901887 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dea4e7-b387-4482-97c4-72248a4c6c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:45.902502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dea4e7-b387-4482-97c4-72248a4c6c87 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:45.902785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dea4e7-b387-4482-97c4-72248a4c6c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:45.903035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dea4e7-b387-4482-97c4-72248a4c6c87 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:45.903710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dea4e7-b387-4482-97c4-72248a4c6c87 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:45.904007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dea4e7-b387-4482-97c4-72248a4c6c87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:45.905503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dea4e7-b387-4482-97c4-72248a4c6c87 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:45.915014 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 456/918] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Sep 23 15:05:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:45.920621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dea4e7-b387-4482-97c4-72248a4c6c87 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:45.932645 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd27b3-1395-4c5d-b114-1024edb9e0d3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:45.973634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85977af8-d280-4633-9edf-44b30deffab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:45.974703 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85977af8-d280-4633-9edf-44b30deffab6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:45.975244 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85977af8-d280-4633-9edf-44b30deffab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:45.975758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85977af8-d280-4633-9edf-44b30deffab6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:45.976571 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85977af8-d280-4633-9edf-44b30deffab6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:45.978696 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85977af8-d280-4633-9edf-44b30deffab6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:45.981844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-85977af8-d280-4633-9edf-44b30deffab6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:45.991619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:45.992838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 15:05:45.993817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:46.000964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 463/919] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:05:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:46.002985 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85977af8-d280-4633-9edf-44b30deffab6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:46.031369 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dea4e7-b387-4482-97c4-72248a4c6c87 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:46.070889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae297b-1ece-4d6d-b7b9-f76ed06f2986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:46.071827 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae297b-1ece-4d6d-b7b9-f76ed06f2986 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:46.071938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:46.072174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae297b-1ece-4d6d-b7b9-f76ed06f2986 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:46.072443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae297b-1ece-4d6d-b7b9-f76ed06f2986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:46.084290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae297b-1ece-4d6d-b7b9-f76ed06f2986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:46.085751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae297b-1ece-4d6d-b7b9-f76ed06f2986 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-7fae297b-1ece-4d6d-b7b9-f76ed06f2986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:46.100801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:46.101781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:46.103383 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:46.103383 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:46.103383 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:46.107122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 457/920] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:46.108272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae297b-1ece-4d6d-b7b9-f76ed06f2986 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:46.121875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85977af8-d280-4633-9edf-44b30deffab6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:46.159860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef3e8d-6e14-4d7e-890f-03aa5be92dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:46.160512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef3e8d-6e14-4d7e-890f-03aa5be92dbd None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:46.160874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef3e8d-6e14-4d7e-890f-03aa5be92dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:46.161141 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef3e8d-6e14-4d7e-890f-03aa5be92dbd None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:46.161771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef3e8d-6e14-4d7e-890f-03aa5be92dbd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:46.162067 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef3e8d-6e14-4d7e-890f-03aa5be92dbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:46.163429 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef3e8d-6e14-4d7e-890f-03aa5be92dbd None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:46.168949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 464/921] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:05:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:46.174788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef3e8d-6e14-4d7e-890f-03aa5be92dbd None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:46.225046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae297b-1ece-4d6d-b7b9-f76ed06f2986 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:46.227999 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:46.261794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee66cb2-550c-4197-80bc-e909f0cc8451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:46.262492 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee66cb2-550c-4197-80bc-e909f0cc8451 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:46.262851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee66cb2-550c-4197-80bc-e909f0cc8451 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:46.263182 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee66cb2-550c-4197-80bc-e909f0cc8451 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:46.263891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee66cb2-550c-4197-80bc-e909f0cc8451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:46.264249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee66cb2-550c-4197-80bc-e909f0cc8451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:46.265729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee66cb2-550c-4197-80bc-e909f0cc8451 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:46.282936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 458/922] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:46.283939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee66cb2-550c-4197-80bc-e909f0cc8451 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:46.311808 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef3e8d-6e14-4d7e-890f-03aa5be92dbd None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:46.335018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:46.364626 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd7b49-415d-489a-a753-b93b8808509a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:46.365455 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd7b49-415d-489a-a753-b93b8808509a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:46.365799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd7b49-415d-489a-a753-b93b8808509a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:46.366126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd7b49-415d-489a-a753-b93b8808509a None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:46.367008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd7b49-415d-489a-a753-b93b8808509a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:46.367179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd7b49-415d-489a-a753-b93b8808509a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:46.368896 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd7b49-415d-489a-a753-b93b8808509a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:46.374829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 465/923] 10.222.0.75 () {56 vars in 1114 bytes} [Fri Sep 23 15:05:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:46.380033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd7b49-415d-489a-a753-b93b8808509a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:46.398287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee66cb2-550c-4197-80bc-e909f0cc8451 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:46.435318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc3ff-2b1d-4b3a-9fed-9e7506746cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:46.436083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc3ff-2b1d-4b3a-9fed-9e7506746cef None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:46.436447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc3ff-2b1d-4b3a-9fed-9e7506746cef None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:46.436909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc3ff-2b1d-4b3a-9fed-9e7506746cef None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:46.445111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcc3ff-2b1d-4b3a-9fed-9e7506746cef None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:46.446841 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcc3ff-2b1d-4b3a-9fed-9e7506746cef None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-d4bcc3ff-2b1d-4b3a-9fed-9e7506746cef None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:46.459455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:46.460493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:46.461543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:46.461543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:46.461543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:46.461543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:46.461543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:46.461543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:46.461543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:46.461543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:46.461543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:05:46.461543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:46.465972 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 459/924] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Sep 23 15:05:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:46.467200 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc3ff-2b1d-4b3a-9fed-9e7506746cef None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:46.509772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd7b49-415d-489a-a753-b93b8808509a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:46.517006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:46.557149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21d434-d670-4342-b0c1-cf446f980cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:46.558203 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21d434-d670-4342-b0c1-cf446f980cec None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:46.558627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21d434-d670-4342-b0c1-cf446f980cec None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:46.559073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21d434-d670-4342-b0c1-cf446f980cec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:46.560428 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21d434-d670-4342-b0c1-cf446f980cec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:46.562262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21d434-d670-4342-b0c1-cf446f980cec None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:46.579575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc3ff-2b1d-4b3a-9fed-9e7506746cef None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:46.608663 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 466/925] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:05:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:05:46.644337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb66ca2-d222-4b07-9a25-78a058d8e4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:46.644987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb66ca2-d222-4b07-9a25-78a058d8e4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:46.645305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb66ca2-d222-4b07-9a25-78a058d8e4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:46.645572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb66ca2-d222-4b07-9a25-78a058d8e4e8 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:46.646314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb66ca2-d222-4b07-9a25-78a058d8e4e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:46.646609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb66ca2-d222-4b07-9a25-78a058d8e4e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:46.648123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb66ca2-d222-4b07-9a25-78a058d8e4e8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:46.668504 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 460/926] 10.222.0.75 () {56 vars in 1149 bytes} [Fri Sep 23 15:05:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:46.674489 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21d434-d670-4342-b0c1-cf446f980cec None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:46.719882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:46.798423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21d434-d670-4342-b0c1-cf446f980cec None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:46.835973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80616c0-b228-49f8-917a-89550bd055c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:46.837263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80616c0-b228-49f8-917a-89550bd055c9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:46.837744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80616c0-b228-49f8-917a-89550bd055c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:46.838210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80616c0-b228-49f8-917a-89550bd055c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:46.850024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80616c0-b228-49f8-917a-89550bd055c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:46.851767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80616c0-b228-49f8-917a-89550bd055c9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-e80616c0-b228-49f8-917a-89550bd055c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:46.867809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:46.868899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:46.869908 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:46.869908 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:46.869908 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:46.870386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 467/927] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:46.876019 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb66ca2-d222-4b07-9a25-78a058d8e4e8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:46.959955 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:46.984187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb66ca2-d222-4b07-9a25-78a058d8e4e8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:47.026047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb3c64-35db-401a-accc-aff6c9631e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:47.026681 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb3c64-35db-401a-accc-aff6c9631e6f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:47.026952 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb3c64-35db-401a-accc-aff6c9631e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:47.027264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb3c64-35db-401a-accc-aff6c9631e6f None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:47.027921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb3c64-35db-401a-accc-aff6c9631e6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:47.028217 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb3c64-35db-401a-accc-aff6c9631e6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:47.029673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb3c64-35db-401a-accc-aff6c9631e6f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:47.043218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 461/928] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:47.048617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80616c0-b228-49f8-917a-89550bd055c9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:47.147914 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:47.162485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80616c0-b228-49f8-917a-89550bd055c9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:47.199319 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b9a06-73a0-4429-a374-e4e826614159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:47.200353 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b9a06-73a0-4429-a374-e4e826614159 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:47.200861 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b9a06-73a0-4429-a374-e4e826614159 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:47.201328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b9a06-73a0-4429-a374-e4e826614159 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:47.208676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b9a06-73a0-4429-a374-e4e826614159 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:47.210444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b9a06-73a0-4429-a374-e4e826614159 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-a35b9a06-73a0-4429-a374-e4e826614159 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:47.224642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:47.225584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:47.226501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:47.226501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:47.226501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:47.226501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:47.226501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:47.226501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:47.226501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:47.226501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:47.228926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 468/929] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:47.240148 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb3c64-35db-401a-accc-aff6c9631e6f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:47.350303 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb3c64-35db-401a-accc-aff6c9631e6f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:47.394933 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273bc6d-73ea-4597-9f83-ccd9edad06a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:47.395540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273bc6d-73ea-4597-9f83-ccd9edad06a5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:47.395813 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273bc6d-73ea-4597-9f83-ccd9edad06a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:47.396080 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273bc6d-73ea-4597-9f83-ccd9edad06a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:47.396796 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273bc6d-73ea-4597-9f83-ccd9edad06a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:47.397096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273bc6d-73ea-4597-9f83-ccd9edad06a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:47.398505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273bc6d-73ea-4597-9f83-ccd9edad06a5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:47.406460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:47.413231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 462/930] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:47.419050 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b9a06-73a0-4429-a374-e4e826614159 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:47.455619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef23e5-af46-4cc6-8c5c-82c9972859c2 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:47.456206 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef23e5-af46-4cc6-8c5c-82c9972859c2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:47.456686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef23e5-af46-4cc6-8c5c-82c9972859c2 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:47.458603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 463/931] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:05:47.465204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622561f-694b-4cdf-af82-5759c691518a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:47.465888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622561f-694b-4cdf-af82-5759c691518a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:47.466363 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622561f-694b-4cdf-af82-5759c691518a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:47.486321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6622561f-694b-4cdf-af82-5759c691518a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:47.487873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6622561f-694b-4cdf-af82-5759c691518a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:47.551386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b9a06-73a0-4429-a374-e4e826614159 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:47.589744 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 464/932] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:47.601992 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba62af-528d-4403-8d07-68725fdab754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:47.602960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba62af-528d-4403-8d07-68725fdab754 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:47.603387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba62af-528d-4403-8d07-68725fdab754 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:47.603799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba62af-528d-4403-8d07-68725fdab754 None admin] PATH_INFO: `/v3/projects/27ac51280046479e82faaf3c0b550f72/users/17ff1e357746447a86d47d19daa18cf5/roles/ea4eebb7bada4fe4a89fa7f71f4b09ce` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:47.606462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba62af-528d-4403-8d07-68725fdab754 None admin] RBAC: Authorizing `identity:create_grant(project_id=27ac51280046479e82faaf3c0b550f72, user_id=17ff1e357746447a86d47d19daa18cf5, role_id=ea4eebb7bada4fe4a89fa7f71f4b09ce)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:47.608143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba62af-528d-4403-8d07-68725fdab754 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:47.625180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 469/933] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:05:47 2022] PUT /identity/v3/projects/27ac51280046479e82faaf3c0b550f72/users/17ff1e357746447a86d47d19daa18cf5/roles/ea4eebb7bada4fe4a89fa7f71f4b09ce => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:05:47.865551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622561f-694b-4cdf-af82-5759c691518a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:47.975735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622561f-694b-4cdf-af82-5759c691518a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:48.005038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:48.016500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb9be4-9832-40d5-b472-01bc587193ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:48.017159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb9be4-9832-40d5-b472-01bc587193ce None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:48.017429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb9be4-9832-40d5-b472-01bc587193ce None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:48.017693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb9be4-9832-40d5-b472-01bc587193ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:48.018279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb9be4-9832-40d5-b472-01bc587193ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:48.019635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb9be4-9832-40d5-b472-01bc587193ce None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:48.026745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e5a53-658e-47bc-9186-af766acd2ecb None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:48.027116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e5a53-658e-47bc-9186-af766acd2ecb None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:48.027287 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e5a53-658e-47bc-9186-af766acd2ecb None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:48.028930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 470/934] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-d3fb9be4-9832-40d5-b472-01bc587193ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:48.029400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:48.030362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:48.031261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 465/935] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Sep 23 15:05:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:48.034027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef044f7-8cb7-413e-88a4-597dc82d5822 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:48.034308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef044f7-8cb7-413e-88a4-597dc82d5822 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:48.034579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef044f7-8cb7-413e-88a4-597dc82d5822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:48.038626 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb9be4-9832-40d5-b472-01bc587193ce None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:48.057761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0ef044f7-8cb7-413e-88a4-597dc82d5822 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:48.059946 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0ef044f7-8cb7-413e-88a4-597dc82d5822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:48.063926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:48.149745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb9be4-9832-40d5-b472-01bc587193ce None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:48.152850 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 471/936] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Sep 23 15:05:48.196670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f4e28-8094-46a9-b0e7-27a74be1492e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:48.197284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f4e28-8094-46a9-b0e7-27a74be1492e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:48.197557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f4e28-8094-46a9-b0e7-27a74be1492e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:48.197829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f4e28-8094-46a9-b0e7-27a74be1492e None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:48.198469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f4e28-8094-46a9-b0e7-27a74be1492e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:48.198769 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f4e28-8094-46a9-b0e7-27a74be1492e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:48.200117 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f4e28-8094-46a9-b0e7-27a74be1492e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:48.205573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 466/937] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Sep 23 15:05:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:48.211297 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef044f7-8cb7-413e-88a4-597dc82d5822 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:48.313529 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef044f7-8cb7-413e-88a4-597dc82d5822 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:48.352906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfa3b5-0045-474d-9886-77e6b40adc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:48.353949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfa3b5-0045-474d-9886-77e6b40adc79 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:48.354421 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfa3b5-0045-474d-9886-77e6b40adc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:48.354906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dfa3b5-0045-474d-9886-77e6b40adc79 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:48.355892 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfa3b5-0045-474d-9886-77e6b40adc79 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:48.356402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfa3b5-0045-474d-9886-77e6b40adc79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:48.357987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dfa3b5-0045-474d-9886-77e6b40adc79 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:48.364310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 472/938] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Sep 23 15:05:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:48.375914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f4e28-8094-46a9-b0e7-27a74be1492e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:48.439636 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:48.441899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfa3b5-0045-474d-9886-77e6b40adc79 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:48.446383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:48.492737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f4e28-8094-46a9-b0e7-27a74be1492e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:48.530423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6acc76-168f-4d64-9993-fd67109c154e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:48.531055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6acc76-168f-4d64-9993-fd67109c154e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:48.531329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6acc76-168f-4d64-9993-fd67109c154e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:48.531594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6acc76-168f-4d64-9993-fd67109c154e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:48.532298 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6acc76-168f-4d64-9993-fd67109c154e None admin] RBAC: Adding query filter params (interface=public, service_id=929b70a2555e455c9d3f3e0120e425f6, region_id=RegionOne) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:48.532618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6acc76-168f-4d64-9993-fd67109c154e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:48.535022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6acc76-168f-4d64-9993-fd67109c154e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:48.552184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 467/939] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Sep 23 15:05:48 2022] GET /identity/v3/endpoints?service_id=929b70a2555e455c9d3f3e0120e425f6&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:48.553960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6acc76-168f-4d64-9993-fd67109c154e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:48.589297 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dfa3b5-0045-474d-9886-77e6b40adc79 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:48.641413 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d540c-a8d0-4fb9-97be-0bcdbaff6142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:48.642072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d540c-a8d0-4fb9-97be-0bcdbaff6142 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:48.642347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d540c-a8d0-4fb9-97be-0bcdbaff6142 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:48.642614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d540c-a8d0-4fb9-97be-0bcdbaff6142 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:48.643210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d540c-a8d0-4fb9-97be-0bcdbaff6142 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:48.645378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d540c-a8d0-4fb9-97be-0bcdbaff6142 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-717d540c-a8d0-4fb9-97be-0bcdbaff6142 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:48.657716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:48.658755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:48.660251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 473/940] 10.222.0.75 () {56 vars in 1132 bytes} [Fri Sep 23 15:05:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:48.664576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d540c-a8d0-4fb9-97be-0bcdbaff6142 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:48.676549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6acc76-168f-4d64-9993-fd67109c154e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:48.697495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:48.712225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccba2b9-b83e-467c-accf-dc09006fc2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:48.713070 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccba2b9-b83e-467c-accf-dc09006fc2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:48.713443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccba2b9-b83e-467c-accf-dc09006fc2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:48.713843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccba2b9-b83e-467c-accf-dc09006fc2e6 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:48.714549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccba2b9-b83e-467c-accf-dc09006fc2e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:48.715998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccba2b9-b83e-467c-accf-dc09006fc2e6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:48.728622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 468/941] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Sep 23 15:05:48 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:05:48.765930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:48.788171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d540c-a8d0-4fb9-97be-0bcdbaff6142 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:48.825681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440a0f5-c9a7-4aac-99f3-e0171817922c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:48.826302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440a0f5-c9a7-4aac-99f3-e0171817922c None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:48.826573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440a0f5-c9a7-4aac-99f3-e0171817922c None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:48.826839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440a0f5-c9a7-4aac-99f3-e0171817922c None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:48.827483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8440a0f5-c9a7-4aac-99f3-e0171817922c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:48.827778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8440a0f5-c9a7-4aac-99f3-e0171817922c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:48.829212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8440a0f5-c9a7-4aac-99f3-e0171817922c None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:48.835096 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 474/942] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Sep 23 15:05:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:48.840863 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccba2b9-b83e-467c-accf-dc09006fc2e6 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:48.950223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccba2b9-b83e-467c-accf-dc09006fc2e6 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:48.986751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf27daa-b352-40bc-a6df-fa15ec62622a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:48.987363 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf27daa-b352-40bc-a6df-fa15ec62622a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:48.987632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf27daa-b352-40bc-a6df-fa15ec62622a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:48.987903 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf27daa-b352-40bc-a6df-fa15ec62622a None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:48.988539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf27daa-b352-40bc-a6df-fa15ec62622a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:48.988913 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf27daa-b352-40bc-a6df-fa15ec62622a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:48.990238 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf27daa-b352-40bc-a6df-fa15ec62622a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:48.995783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 469/943] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Sep 23 15:05:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:49.002158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440a0f5-c9a7-4aac-99f3-e0171817922c None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:49.062402 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cf48b-08f1-4e57-8620-c5ca90e31c48 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:49.062797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cf48b-08f1-4e57-8620-c5ca90e31c48 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:49.063426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cf48b-08f1-4e57-8620-c5ca90e31c48 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:49.066384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 470/944] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:05:49.071518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756bd82-85b2-4b05-9ef4-c3a1912c28ff None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:49.071768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756bd82-85b2-4b05-9ef4-c3a1912c28ff None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:49.072047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756bd82-85b2-4b05-9ef4-c3a1912c28ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:49.096244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a756bd82-85b2-4b05-9ef4-c3a1912c28ff None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:49.097690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a756bd82-85b2-4b05-9ef4-c3a1912c28ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:49.116126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440a0f5-c9a7-4aac-99f3-e0171817922c None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:49.147215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:49.153710 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808a4fa-99c1-4488-9b5b-0ab7dd1d40ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:49.154552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808a4fa-99c1-4488-9b5b-0ab7dd1d40ab None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:49.154968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808a4fa-99c1-4488-9b5b-0ab7dd1d40ab None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:49.155524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808a4fa-99c1-4488-9b5b-0ab7dd1d40ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:49.156348 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808a4fa-99c1-4488-9b5b-0ab7dd1d40ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:49.157888 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808a4fa-99c1-4488-9b5b-0ab7dd1d40ab None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:49.180629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 471/945] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:49.180629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:49.185078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 475/946] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:05:49 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:05:49.496484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756bd82-85b2-4b05-9ef4-c3a1912c28ff None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:49.606366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756bd82-85b2-4b05-9ef4-c3a1912c28ff None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:49.646908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3696ba-35cf-41ef-889b-cecc6ba63c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:49.647880 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3696ba-35cf-41ef-889b-cecc6ba63c44 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:49.648351 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3696ba-35cf-41ef-889b-cecc6ba63c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:49.648906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3696ba-35cf-41ef-889b-cecc6ba63c44 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:49.649895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3696ba-35cf-41ef-889b-cecc6ba63c44 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:49.651516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3696ba-35cf-41ef-889b-cecc6ba63c44 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:49.654380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: WARNING keystone.server.flask.common [None req-8e3696ba-35cf-41ef-889b-cecc6ba63c44 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 23 15:05:49.673640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.sql.core [None req-8e3696ba-35cf-41ef-889b-cecc6ba63c44 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 23 15:05:49.673640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 23 15:05:49.673640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [parameters: {'id': 'd38377460b204c269e2132b52fa20773', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 23 15:05:49.673640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81865) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 23 15:05:49.692515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-8e3696ba-35cf-41ef-889b-cecc6ba63c44 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d38377460b204c269e2132b52fa20773.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d38377460b204c269e2132b52fa20773. Sep 23 15:05:49.692515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:49.692515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 15:05:49.692515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 15:05:49.692515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 15:05:49.692515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 15:05:49.692515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 15:05:49.692515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 15:05:49.692515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 15:05:49.692515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application conn.query(q) Sep 23 15:05:49.692515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 15:05:49.692515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 15:05:49.692515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result.read() Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 23 15:05:49.693426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application next(self.gen) Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application yield resource Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application next(self.gen) Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.session.rollback() Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.force_reraise() Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise self.value Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application session.commit() Sep 23 15:05:49.694399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._prepare_impl() Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.session.flush() Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._flush(objects) Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application compat.raise_( Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application flush_context.execute() Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 23 15:05:49.695278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rec.execute(self) Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 23 15:05:49.697329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application conn.query(q) Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result.read() Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 15:05:49.699332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [parameters: {'id': 'd38377460b204c269e2132b52fa20773', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:49.701479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:49.703304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:49.705418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:49.705418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 23 15:05:49.705418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 23 15:05:49.705418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:49.705418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:49.705418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 23 15:05:49.705418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 23 15:05:49.705418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d38377460b204c269e2132b52fa20773. Sep 23 15:05:49.705418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:49.708245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 472/947] 10.222.0.75 () {60 vars in 1140 bytes} [Fri Sep 23 15:05:49 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:49.714210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808a4fa-99c1-4488-9b5b-0ab7dd1d40ab None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:49.822494 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808a4fa-99c1-4488-9b5b-0ab7dd1d40ab None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:49.859317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba456f2-e25c-4588-8935-fc8f801924de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:49.860332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba456f2-e25c-4588-8935-fc8f801924de None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:49.860828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba456f2-e25c-4588-8935-fc8f801924de None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:49.861292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba456f2-e25c-4588-8935-fc8f801924de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:49.874285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba456f2-e25c-4588-8935-fc8f801924de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:49.876347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba456f2-e25c-4588-8935-fc8f801924de None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-fba456f2-e25c-4588-8935-fc8f801924de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:49.891298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:49.892256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:49.893168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:49.893168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:49.893168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:49.893168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:49.893168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:49.893168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:49.893168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:49.893168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:49.897051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 476/948] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:49.899617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3696ba-35cf-41ef-889b-cecc6ba63c44 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:49.931665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:49.995957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd8d9f-26cb-4231-aa1f-65273de1b546 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:49.996307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd8d9f-26cb-4231-aa1f-65273de1b546 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:49.996496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd8d9f-26cb-4231-aa1f-65273de1b546 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:49.998317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 477/949] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:50.003159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d60e8-2119-415a-a6b6-0b667fb0ce21 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:50.003434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d60e8-2119-415a-a6b6-0b667fb0ce21 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:50.003700 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d60e8-2119-415a-a6b6-0b667fb0ce21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:50.008177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3696ba-35cf-41ef-889b-cecc6ba63c44 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:50.022920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-8c7d60e8-2119-415a-a6b6-0b667fb0ce21 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:50.024821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8c7d60e8-2119-415a-a6b6-0b667fb0ce21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:50.045029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04adf252-21dc-4c2c-a5aa-d47f2a1efdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:50.045646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adf252-21dc-4c2c-a5aa-d47f2a1efdb7 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:50.045922 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adf252-21dc-4c2c-a5aa-d47f2a1efdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:50.046191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adf252-21dc-4c2c-a5aa-d47f2a1efdb7 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:50.046873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04adf252-21dc-4c2c-a5aa-d47f2a1efdb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:50.047172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04adf252-21dc-4c2c-a5aa-d47f2a1efdb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:50.048829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04adf252-21dc-4c2c-a5aa-d47f2a1efdb7 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:50.070231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 473/950] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Sep 23 15:05:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:50.085203 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:50.107961 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 478/951] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:50.384801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04adf252-21dc-4c2c-a5aa-d47f2a1efdb7 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:50.476035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:50.486202 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04adf252-21dc-4c2c-a5aa-d47f2a1efdb7 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:50.522051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b930c-6908-4eef-a346-bfcec64e57ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:50.522723 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b930c-6908-4eef-a346-bfcec64e57ab None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:50.522994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b930c-6908-4eef-a346-bfcec64e57ab None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:50.523209 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b930c-6908-4eef-a346-bfcec64e57ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:50.534274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b930c-6908-4eef-a346-bfcec64e57ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-fa0b930c-6908-4eef-a346-bfcec64e57ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:05:50.539086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:50.540846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:50.541717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:50.541717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:50.541717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:50.541717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:50.541717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:05:50.541717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:05:50.541717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:05:50.541717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:05:50.541717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:05:50.541717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:05:50.541717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:05:50.541717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:50.541717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 474/952] 10.222.0.75 () {56 vars in 1120 bytes} [Fri Sep 23 15:05:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:50.546862 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d60e8-2119-415a-a6b6-0b667fb0ce21 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:50.650296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d60e8-2119-415a-a6b6-0b667fb0ce21 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:50.687391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a758be3-fe51-492e-96e1-d3f3ddde0971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:50.688001 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a758be3-fe51-492e-96e1-d3f3ddde0971 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:50.688274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a758be3-fe51-492e-96e1-d3f3ddde0971 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:50.688541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a758be3-fe51-492e-96e1-d3f3ddde0971 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:50.689267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a758be3-fe51-492e-96e1-d3f3ddde0971 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:50.689565 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a758be3-fe51-492e-96e1-d3f3ddde0971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:50.690933 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a758be3-fe51-492e-96e1-d3f3ddde0971 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:50.699858 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 479/953] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Sep 23 15:05:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:50.706177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b930c-6908-4eef-a346-bfcec64e57ab None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:50.810661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b930c-6908-4eef-a346-bfcec64e57ab None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:50.845649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95591396-a4f3-4401-886f-4c7c2913e0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:50.846272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95591396-a4f3-4401-886f-4c7c2913e0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:50.846544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95591396-a4f3-4401-886f-4c7c2913e0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:50.846811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95591396-a4f3-4401-886f-4c7c2913e0f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:50.857314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95591396-a4f3-4401-886f-4c7c2913e0f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:50.859073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95591396-a4f3-4401-886f-4c7c2913e0f0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-95591396-a4f3-4401-886f-4c7c2913e0f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:50.873767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:50.874782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:50.876196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:50.876196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:50.876196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:50.876196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 475/954] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:50.880799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a758be3-fe51-492e-96e1-d3f3ddde0971 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:50.929398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:50.990757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a758be3-fe51-492e-96e1-d3f3ddde0971 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:51.029782 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edf95b-0a7c-4866-a2d7-10d478d92f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:51.030768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edf95b-0a7c-4866-a2d7-10d478d92f04 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:51.031267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edf95b-0a7c-4866-a2d7-10d478d92f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:51.031726 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edf95b-0a7c-4866-a2d7-10d478d92f04 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:51.032567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7edf95b-0a7c-4866-a2d7-10d478d92f04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:51.033129 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7edf95b-0a7c-4866-a2d7-10d478d92f04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:51.034737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7edf95b-0a7c-4866-a2d7-10d478d92f04 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:51.040723 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442de46c-3236-4f33-841f-600e2617974c None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:51.041006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442de46c-3236-4f33-841f-600e2617974c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:51.041284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442de46c-3236-4f33-841f-600e2617974c None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:51.043549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 476/955] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:05:51.048189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471f729-8152-44ba-9fc9-64cd8004ea2d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:51.048467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471f729-8152-44ba-9fc9-64cd8004ea2d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:51.048769 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471f729-8152-44ba-9fc9-64cd8004ea2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:51.049684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 480/956] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:51.057390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edf95b-0a7c-4866-a2d7-10d478d92f04 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:51.058699 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:51.074432 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5471f729-8152-44ba-9fc9-64cd8004ea2d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:51.076915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5471f729-8152-44ba-9fc9-64cd8004ea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:51.164226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:51.170678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edf95b-0a7c-4866-a2d7-10d478d92f04 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:51.176801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 477/957] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:51.215612 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aba4c-2a96-4669-ac37-8d81976d44c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:51.216168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aba4c-2a96-4669-ac37-8d81976d44c3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:51.216442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aba4c-2a96-4669-ac37-8d81976d44c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:51.216750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aba4c-2a96-4669-ac37-8d81976d44c3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:51.225026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123aba4c-2a96-4669-ac37-8d81976d44c3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:51.226469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123aba4c-2a96-4669-ac37-8d81976d44c3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-123aba4c-2a96-4669-ac37-8d81976d44c3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:51.239218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:51.240177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:51.241556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:51.241556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:51.241556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:51.241556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:51.241556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:51.241556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:51.241556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:51.241556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:51.241556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 15:05:51.241556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:51.241556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 481/958] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Sep 23 15:05:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:51.246701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471f729-8152-44ba-9fc9-64cd8004ea2d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:51.353640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471f729-8152-44ba-9fc9-64cd8004ea2d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:51.385077 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:51.391220 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf29d5-5382-4f78-b6da-ae156026c5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:51.392200 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf29d5-5382-4f78-b6da-ae156026c5ee None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:51.392570 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf29d5-5382-4f78-b6da-ae156026c5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:51.392982 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf29d5-5382-4f78-b6da-ae156026c5ee None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:51.393823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf29d5-5382-4f78-b6da-ae156026c5ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:51.394226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf29d5-5382-4f78-b6da-ae156026c5ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:51.395812 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf29d5-5382-4f78-b6da-ae156026c5ee None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:51.415133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 478/959] 10.222.0.75 () {56 vars in 1149 bytes} [Fri Sep 23 15:05:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:51.422126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aba4c-2a96-4669-ac37-8d81976d44c3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:51.478110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf29d5-5382-4f78-b6da-ae156026c5ee None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:51.564617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aba4c-2a96-4669-ac37-8d81976d44c3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:51.600644 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b1826-6670-4bf6-928c-f26407eef7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:51.601247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b1826-6670-4bf6-928c-f26407eef7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:51.601517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b1826-6670-4bf6-928c-f26407eef7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:51.601775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b1826-6670-4bf6-928c-f26407eef7a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:51.605257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf29d5-5382-4f78-b6da-ae156026c5ee None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:51.632306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:51.644378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2952baa-b989-4f29-99c7-9d03a4ae427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:51.645091 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2952baa-b989-4f29-99c7-9d03a4ae427f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:51.645333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2952baa-b989-4f29-99c7-9d03a4ae427f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:51.645566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2952baa-b989-4f29-99c7-9d03a4ae427f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:51.646145 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2952baa-b989-4f29-99c7-9d03a4ae427f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:51.647483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2952baa-b989-4f29-99c7-9d03a4ae427f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-e2952baa-b989-4f29-99c7-9d03a4ae427f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:51.656430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:51.657668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:51.658523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 479/960] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Sep 23 15:05:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:51.659349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0e652-df61-4704-891d-21e5afe4c892 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:51.659349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0e652-df61-4704-891d-21e5afe4c892 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:51.659349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0e652-df61-4704-891d-21e5afe4c892 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:51.660486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 480/961] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:51.665355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0e652-df61-4704-891d-21e5afe4c892 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:51.682183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b1826-6670-4bf6-928c-f26407eef7a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:51.683714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b1826-6670-4bf6-928c-f26407eef7a7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:51.725370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:51.775867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0e652-df61-4704-891d-21e5afe4c892 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:51.784997 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 482/962] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:05:51 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:05:51.786415 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543a337-c994-4aa7-b285-7bd2f29d745b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:51.786826 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543a337-c994-4aa7-b285-7bd2f29d745b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:51.787250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543a337-c994-4aa7-b285-7bd2f29d745b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:51.806835 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b543a337-c994-4aa7-b285-7bd2f29d745b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:51.808201 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b543a337-c994-4aa7-b285-7bd2f29d745b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:51.818510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab2fc3-758e-4c98-be6a-fda873ac8f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:51.819501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab2fc3-758e-4c98-be6a-fda873ac8f53 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:51.819965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab2fc3-758e-4c98-be6a-fda873ac8f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:51.820467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab2fc3-758e-4c98-be6a-fda873ac8f53 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:51.821407 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab2fc3-758e-4c98-be6a-fda873ac8f53 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:51.821878 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab2fc3-758e-4c98-be6a-fda873ac8f53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:51.823436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab2fc3-758e-4c98-be6a-fda873ac8f53 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:51.834118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 481/963] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Sep 23 15:05:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:51.834118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab2fc3-758e-4c98-be6a-fda873ac8f53 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:51.873858 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:51.889205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 483/964] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:51.889844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543a337-c994-4aa7-b285-7bd2f29d745b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:51.974077 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab2fc3-758e-4c98-be6a-fda873ac8f53 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:51.998483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:52.022415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927772f-88d6-4109-87ee-4e67c2d5ec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:52.023043 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927772f-88d6-4109-87ee-4e67c2d5ec2d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:52.023316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927772f-88d6-4109-87ee-4e67c2d5ec2d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:52.023585 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927772f-88d6-4109-87ee-4e67c2d5ec2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:52.026420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543a337-c994-4aa7-b285-7bd2f29d745b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:52.038685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927772f-88d6-4109-87ee-4e67c2d5ec2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:52.040222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927772f-88d6-4109-87ee-4e67c2d5ec2d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-b927772f-88d6-4109-87ee-4e67c2d5ec2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:05:52.055508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:52.057060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:05:52.057946 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:05:52.057946 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:05:52.057946 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:52.057946 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 482/965] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:52.062007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927772f-88d6-4109-87ee-4e67c2d5ec2d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:52.068895 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18194e-9a21-4e51-8acb-a0790c2770d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:52.069730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18194e-9a21-4e51-8acb-a0790c2770d1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:52.070146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18194e-9a21-4e51-8acb-a0790c2770d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:52.070552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18194e-9a21-4e51-8acb-a0790c2770d1 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:52.071790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18194e-9a21-4e51-8acb-a0790c2770d1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:52.072333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18194e-9a21-4e51-8acb-a0790c2770d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:52.073926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e18194e-9a21-4e51-8acb-a0790c2770d1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:52.084655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 484/966] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Sep 23 15:05:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:52.086047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18194e-9a21-4e51-8acb-a0790c2770d1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:52.148818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:52.178444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927772f-88d6-4109-87ee-4e67c2d5ec2d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:52.201450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18194e-9a21-4e51-8acb-a0790c2770d1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:52.226171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb318e-c499-4040-940b-d907ceb2d3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:52.227184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb318e-c499-4040-940b-d907ceb2d3ca None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:52.227575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb318e-c499-4040-940b-d907ceb2d3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:52.228007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeb318e-c499-4040-940b-d907ceb2d3ca None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:52.229186 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb318e-c499-4040-940b-d907ceb2d3ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:52.229991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb318e-c499-4040-940b-d907ceb2d3ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:52.231045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeb318e-c499-4040-940b-d907ceb2d3ca None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:52.240812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef49e0d-938e-442c-ac56-971d26c1c715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:52.242049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef49e0d-938e-442c-ac56-971d26c1c715 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:52.242162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef49e0d-938e-442c-ac56-971d26c1c715 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:52.242550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef49e0d-938e-442c-ac56-971d26c1c715 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:52.243678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef49e0d-938e-442c-ac56-971d26c1c715 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:52.245282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef49e0d-938e-442c-ac56-971d26c1c715 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:52.246644 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 483/967] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:52.247801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb318e-c499-4040-940b-d907ceb2d3ca None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:52.271965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 485/968] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:05:52 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:05:52.273140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef49e0d-938e-442c-ac56-971d26c1c715 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:52.294651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:52.388570 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeb318e-c499-4040-940b-d907ceb2d3ca None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:52.417278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef49e0d-938e-442c-ac56-971d26c1c715 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:52.427995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:52.435247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad8a28-a314-45bb-897d-fdec5d5431bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:52.435900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad8a28-a314-45bb-897d-fdec5d5431bc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:52.436229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad8a28-a314-45bb-897d-fdec5d5431bc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:52.436442 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad8a28-a314-45bb-897d-fdec5d5431bc None admin] PATH_INFO: `/v3/services/compute` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:52.437401 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad8a28-a314-45bb-897d-fdec5d5431bc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:52.438599 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad8a28-a314-45bb-897d-fdec5d5431bc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-dfad8a28-a314-45bb-897d-fdec5d5431bc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:52.451365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:52.452558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:52.454155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 484/969] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:52.454956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec2f40-0059-4424-951a-ce2b3386b177 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:52.454956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec2f40-0059-4424-951a-ce2b3386b177 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:52.454956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec2f40-0059-4424-951a-ce2b3386b177 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:52.460351 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 485/970] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:52.461100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec2f40-0059-4424-951a-ce2b3386b177 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:52.467668 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f7d47-a553-4164-a27b-9da0d7a7544d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:52.468390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f7d47-a553-4164-a27b-9da0d7a7544d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:52.468698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f7d47-a553-4164-a27b-9da0d7a7544d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:52.468969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f7d47-a553-4164-a27b-9da0d7a7544d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:52.476635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f7d47-a553-4164-a27b-9da0d7a7544d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:52.477871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f7d47-a553-4164-a27b-9da0d7a7544d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-d08f7d47-a553-4164-a27b-9da0d7a7544d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:05:52.489872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:52.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:52.491622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:05:52.491622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:05:52.491622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:05:52.491622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:05:52.491622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:05:52.491622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:05:52.491622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:05:52.491622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:52.491622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 486/971] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:52.492127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d0af-a39f-4487-9988-b974627dda50 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:52.492460 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d0af-a39f-4487-9988-b974627dda50 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:52.492970 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d0af-a39f-4487-9988-b974627dda50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:52.512761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-09c8d0af-a39f-4487-9988-b974627dda50 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:52.514387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-09c8d0af-a39f-4487-9988-b974627dda50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:52.582646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:52.590681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 487/972] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:52.591604 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8d0af-a39f-4487-9988-b974627dda50 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:52.602775 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec2f40-0059-4424-951a-ce2b3386b177 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:52.642606 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0a4f8-2d7a-4a66-9d4c-50e4a9cbdb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:52.643285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0a4f8-2d7a-4a66-9d4c-50e4a9cbdb4d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:52.643654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0a4f8-2d7a-4a66-9d4c-50e4a9cbdb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:52.643980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0a4f8-2d7a-4a66-9d4c-50e4a9cbdb4d None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:52.644741 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0a4f8-2d7a-4a66-9d4c-50e4a9cbdb4d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:52.645102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0a4f8-2d7a-4a66-9d4c-50e4a9cbdb4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:52.646513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0a4f8-2d7a-4a66-9d4c-50e4a9cbdb4d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:52.652647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 486/973] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Sep 23 15:05:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:52.657990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0a4f8-2d7a-4a66-9d4c-50e4a9cbdb4d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:52.704301 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:52.711166 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8d0af-a39f-4487-9988-b974627dda50 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:52.747142 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca677f-cee8-474e-b096-8e88fc1bc7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:52.748180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca677f-cee8-474e-b096-8e88fc1bc7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:52.748720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca677f-cee8-474e-b096-8e88fc1bc7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:52.749206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca677f-cee8-474e-b096-8e88fc1bc7c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:52.750199 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca677f-cee8-474e-b096-8e88fc1bc7c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:52.750777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca677f-cee8-474e-b096-8e88fc1bc7c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:52.752396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca677f-cee8-474e-b096-8e88fc1bc7c2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:52.765843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0a4f8-2d7a-4a66-9d4c-50e4a9cbdb4d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:52.766465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 488/974] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:05:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:52.771486 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca677f-cee8-474e-b096-8e88fc1bc7c2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:52.802919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a211cc-d7aa-4020-a759-1495561a4e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:52.803440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a211cc-d7aa-4020-a759-1495561a4e11 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:52.803710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a211cc-d7aa-4020-a759-1495561a4e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:52.803975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a211cc-d7aa-4020-a759-1495561a4e11 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:52.804689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a211cc-d7aa-4020-a759-1495561a4e11 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:52.805016 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a211cc-d7aa-4020-a759-1495561a4e11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:52.806650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a211cc-d7aa-4020-a759-1495561a4e11 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:52.818064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 487/975] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Sep 23 15:05:52 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:52.875987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a211cc-d7aa-4020-a759-1495561a4e11 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:52.890394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca677f-cee8-474e-b096-8e88fc1bc7c2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:52.908343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:52.932192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b516e4c-7de2-45f3-81d5-047bca2635f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:52.932831 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516e4c-7de2-45f3-81d5-047bca2635f9 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:52.933101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516e4c-7de2-45f3-81d5-047bca2635f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:52.933365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516e4c-7de2-45f3-81d5-047bca2635f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:52.934092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b516e4c-7de2-45f3-81d5-047bca2635f9 None admin] RBAC: Adding query filter params (role.id=ea4eebb7bada4fe4a89fa7f71f4b09ce, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=17ff1e357746447a86d47d19daa18cf5) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:52.934391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b516e4c-7de2-45f3-81d5-047bca2635f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:52.935830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b516e4c-7de2-45f3-81d5-047bca2635f9 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:52.942775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 489/976] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:05:52 2022] GET /identity/v3/role_assignments?user.id=17ff1e357746447a86d47d19daa18cf5&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=ea4eebb7bada4fe4a89fa7f71f4b09ce => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:53.016088 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a211cc-d7aa-4020-a759-1495561a4e11 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:53.056546 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9f270-7a6b-4c70-add7-d46e08366c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:53.057101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9f270-7a6b-4c70-add7-d46e08366c79 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:53.057370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9f270-7a6b-4c70-add7-d46e08366c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:53.057634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9f270-7a6b-4c70-add7-d46e08366c79 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:53.058296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9f270-7a6b-4c70-add7-d46e08366c79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:53.059809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9f270-7a6b-4c70-add7-d46e08366c79 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:53.076446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.sql.core [None req-23f9f270-7a6b-4c70-add7-d46e08366c79 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 23 15:05:53.076446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 23 15:05:53.076446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [parameters: {'user_id': 'c9a28bfd6b084bb4a95708d9d7964efb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 23 15:05:53.076446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81865) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-23f9f270-7a6b-4c70-add7-d46e08366c79 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application conn.query(q) Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result.read() Sep 23 15:05:53.084169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 15:05:53.085146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application next(self.gen) Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application yield resource Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application next(self.gen) Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.session.rollback() Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.force_reraise() Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise self.value Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application session.commit() Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 23 15:05:53.086375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._prepare_impl() Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.session.flush() Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._flush(objects) Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application compat.raise_( Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application flush_context.execute() Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rec.execute(self) Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 23 15:05:53.087956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 15:05:53.089046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application conn.query(q) Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application result.read() Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 15:05:53.089909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c9a28bfd6b084bb4a95708d9d7964efb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:53.097325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 23 15:05:53.098356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 23 15:05:53.099460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 23 15:05:53.099460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 23 15:05:53.099460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 23 15:05:53.099460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 23 15:05:53.099460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 23 15:05:53.099460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:53.099460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 488/977] 10.222.0.75 () {60 vars in 1131 bytes} [Fri Sep 23 15:05:52 2022] POST /identity/v3/users => generated 160 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:53.099949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b516e4c-7de2-45f3-81d5-047bca2635f9 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:53.127090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:53.212917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b516e4c-7de2-45f3-81d5-047bca2635f9 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:53.247998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531621c3-3497-433e-8521-e514ae0a0cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:53.248961 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531621c3-3497-433e-8521-e514ae0a0cfc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:53.249378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531621c3-3497-433e-8521-e514ae0a0cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:53.249783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531621c3-3497-433e-8521-e514ae0a0cfc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:53.258651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531621c3-3497-433e-8521-e514ae0a0cfc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:53.260404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531621c3-3497-433e-8521-e514ae0a0cfc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-531621c3-3497-433e-8521-e514ae0a0cfc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:05:53.273161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:53.274478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:53.275617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:53.275617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:53.275617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:05:53.275617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:05:53.275617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:05:53.275617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:05:53.275617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:05:53.275617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:05:53.275617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 15:05:53.275617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:53.277942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 490/978] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:05:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:53.284640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9f270-7a6b-4c70-add7-d46e08366c79 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:53.377338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:53.395377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9f270-7a6b-4c70-add7-d46e08366c79 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:53.429720 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd8d36-e72e-4729-b8e7-5de7baaff86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:53.430400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd8d36-e72e-4729-b8e7-5de7baaff86a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:53.430669 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd8d36-e72e-4729-b8e7-5de7baaff86a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:53.430930 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd8d36-e72e-4729-b8e7-5de7baaff86a None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:53.431639 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd8d36-e72e-4729-b8e7-5de7baaff86a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:53.431933 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd8d36-e72e-4729-b8e7-5de7baaff86a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:53.433612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd8d36-e72e-4729-b8e7-5de7baaff86a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:53.473431 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 489/979] 10.222.0.75 () {56 vars in 1101 bytes} [Fri Sep 23 15:05:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:54.548382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b751e-cf8b-4206-872b-41a30d9c431d None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:54.548893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b751e-cf8b-4206-872b-41a30d9c431d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:54.548970 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b751e-cf8b-4206-872b-41a30d9c431d None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:54.550376 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 491/980] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:54.556097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3e378-7fbe-4f08-9289-837100726c29 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:54.556717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3e378-7fbe-4f08-9289-837100726c29 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:54.556717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3e378-7fbe-4f08-9289-837100726c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:54.576637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7df3e378-7fbe-4f08-9289-837100726c29 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:54.578142 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7df3e378-7fbe-4f08-9289-837100726c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:54.650752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 490/981] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:54.921081 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b751e-cf8b-4206-872b-41a30d9c431d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:55.015785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:55.040289 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b751e-cf8b-4206-872b-41a30d9c431d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:55.083653 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae6a50-aad0-46b7-83eb-3cc7345f2f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:55.084282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae6a50-aad0-46b7-83eb-3cc7345f2f4e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:55.084550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae6a50-aad0-46b7-83eb-3cc7345f2f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:55.084852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae6a50-aad0-46b7-83eb-3cc7345f2f4e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:55.085438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae6a50-aad0-46b7-83eb-3cc7345f2f4e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:55.087448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae6a50-aad0-46b7-83eb-3cc7345f2f4e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-bdae6a50-aad0-46b7-83eb-3cc7345f2f4e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:55.097450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:55.100804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:55.101730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 492/982] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Sep 23 15:05:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:55.107633 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3e378-7fbe-4f08-9289-837100726c29 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:55.128677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c841fc-7081-49f9-b26d-c35831a7c14e None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:55.129041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c841fc-7081-49f9-b26d-c35831a7c14e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:55.129229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c841fc-7081-49f9-b26d-c35831a7c14e None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:55.131346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 493/983] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:55.136371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63076b-96e5-4d37-aec0-7962b06b7a4c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:55.136987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63076b-96e5-4d37-aec0-7962b06b7a4c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:55.136987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63076b-96e5-4d37-aec0-7962b06b7a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:55.162096 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4e63076b-96e5-4d37-aec0-7962b06b7a4c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:55.163290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4e63076b-96e5-4d37-aec0-7962b06b7a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:55.185915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:55.225732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3e378-7fbe-4f08-9289-837100726c29 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:55.252629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 494/984] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:55.266955 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776e3d6-3fde-4ba7-b910-9292cca12be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:55.267865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776e3d6-3fde-4ba7-b910-9292cca12be3 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:55.268345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776e3d6-3fde-4ba7-b910-9292cca12be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:55.268701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776e3d6-3fde-4ba7-b910-9292cca12be3 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:55.269347 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776e3d6-3fde-4ba7-b910-9292cca12be3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:55.269647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776e3d6-3fde-4ba7-b910-9292cca12be3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:55.271013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776e3d6-3fde-4ba7-b910-9292cca12be3 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:55.281674 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 491/985] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Sep 23 15:05:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:55.282814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63076b-96e5-4d37-aec0-7962b06b7a4c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:55.324022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac37e5-d68b-48c4-a498-30ba9d389b30 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:55.324330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac37e5-d68b-48c4-a498-30ba9d389b30 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:55.324955 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac37e5-d68b-48c4-a498-30ba9d389b30 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:55.326476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 492/986] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:55.332001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a6ecc-b3bb-4f3a-ab81-7dc4d7431a6e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:55.332338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a6ecc-b3bb-4f3a-ab81-7dc4d7431a6e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:55.332634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a6ecc-b3bb-4f3a-ab81-7dc4d7431a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:55.358119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-bf5a6ecc-b3bb-4f3a-ab81-7dc4d7431a6e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:55.359632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-bf5a6ecc-b3bb-4f3a-ab81-7dc4d7431a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:55.380319 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:55.411582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63076b-96e5-4d37-aec0-7962b06b7a4c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:55.454635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27a0f1-804e-49a6-965d-7f12b760142e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:55.455716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27a0f1-804e-49a6-965d-7f12b760142e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:55.456209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27a0f1-804e-49a6-965d-7f12b760142e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:55.456678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27a0f1-804e-49a6-965d-7f12b760142e None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:55.457721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27a0f1-804e-49a6-965d-7f12b760142e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:55.458217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27a0f1-804e-49a6-965d-7f12b760142e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:55.460561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27a0f1-804e-49a6-965d-7f12b760142e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:55.463399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 493/987] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:55.469453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 495/988] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Sep 23 15:05:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:55.475223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a6ecc-b3bb-4f3a-ab81-7dc4d7431a6e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:55.530995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27a0f1-804e-49a6-965d-7f12b760142e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:55.565570 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:55.618502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a6ecc-b3bb-4f3a-ab81-7dc4d7431a6e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:55.644818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27a0f1-804e-49a6-965d-7f12b760142e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:55.662219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5c545-bccf-4a65-bc04-26cb6fca637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:55.662820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5c545-bccf-4a65-bc04-26cb6fca637a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:55.663103 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5c545-bccf-4a65-bc04-26cb6fca637a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:55.663368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5c545-bccf-4a65-bc04-26cb6fca637a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:55.664094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5c545-bccf-4a65-bc04-26cb6fca637a None admin] RBAC: Adding query filter params (interface=admin, service_id=929b70a2555e455c9d3f3e0120e425f6, region_id=RegionOne) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:55.664432 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5c545-bccf-4a65-bc04-26cb6fca637a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:55.665815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5c545-bccf-4a65-bc04-26cb6fca637a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:55.677551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 494/989] 10.222.0.75 () {56 vars in 1249 bytes} [Fri Sep 23 15:05:55 2022] GET /identity/v3/endpoints?service_id=929b70a2555e455c9d3f3e0120e425f6&interface=admin®ion_id=RegionOne => generated 193 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:55.678579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5c545-bccf-4a65-bc04-26cb6fca637a None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:55.687152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163114b1-eaf7-42f7-a329-cccedbb94370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:55.687779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163114b1-eaf7-42f7-a329-cccedbb94370 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:55.688072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163114b1-eaf7-42f7-a329-cccedbb94370 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:55.688317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163114b1-eaf7-42f7-a329-cccedbb94370 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:55.689002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163114b1-eaf7-42f7-a329-cccedbb94370 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:55.690412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163114b1-eaf7-42f7-a329-cccedbb94370 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:55.700766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:55.707046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 496/990] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:05:55 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:05:55.739195 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163114b1-eaf7-42f7-a329-cccedbb94370 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:55.805100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5c545-bccf-4a65-bc04-26cb6fca637a None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:55.837524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:55.853786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56680372-782c-48b6-a298-e1c83892a699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:55.854727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56680372-782c-48b6-a298-e1c83892a699 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:55.855001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56680372-782c-48b6-a298-e1c83892a699 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:55.855575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56680372-782c-48b6-a298-e1c83892a699 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:55.856581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56680372-782c-48b6-a298-e1c83892a699 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:55.858212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56680372-782c-48b6-a298-e1c83892a699 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:55.859968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163114b1-eaf7-42f7-a329-cccedbb94370 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:55.866586 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 495/991] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Sep 23 15:05:55 2022] GET /identity/v3/services => generated 1727 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:05:55.904085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e9182-b2e0-473a-9669-fb6f5f3c7547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:55.904757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e9182-b2e0-473a-9669-fb6f5f3c7547 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:55.905027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e9182-b2e0-473a-9669-fb6f5f3c7547 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:55.905286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e9182-b2e0-473a-9669-fb6f5f3c7547 None admin] PATH_INFO: `/v3/services/image` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:55.905873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e9182-b2e0-473a-9669-fb6f5f3c7547 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:55.907283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e9182-b2e0-473a-9669-fb6f5f3c7547 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-1e1e9182-b2e0-473a-9669-fb6f5f3c7547 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:55.917406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:55.918503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:55.919581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 497/992] 10.222.0.75 () {56 vars in 1105 bytes} [Fri Sep 23 15:05:55 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:55.925046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56680372-782c-48b6-a298-e1c83892a699 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:56.043541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56680372-782c-48b6-a298-e1c83892a699 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:56.079688 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96951f5e-d5d1-43f9-892c-8f206444da72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:56.080295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96951f5e-d5d1-43f9-892c-8f206444da72 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:56.080565 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96951f5e-d5d1-43f9-892c-8f206444da72 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:56.080860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96951f5e-d5d1-43f9-892c-8f206444da72 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:56.081496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96951f5e-d5d1-43f9-892c-8f206444da72 None admin] RBAC: Adding query filter params (name=image) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:56.081793 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96951f5e-d5d1-43f9-892c-8f206444da72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:56.083148 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96951f5e-d5d1-43f9-892c-8f206444da72 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:56.089525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 496/993] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:56.094612 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e9182-b2e0-473a-9669-fb6f5f3c7547 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:56.161018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:56.203725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e9182-b2e0-473a-9669-fb6f5f3c7547 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:56.242177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e5c8a-acb1-4d35-ab5b-f9bd556f82dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:56.242840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e5c8a-acb1-4d35-ab5b-f9bd556f82dc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:56.243114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e5c8a-acb1-4d35-ab5b-f9bd556f82dc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:56.243403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e5c8a-acb1-4d35-ab5b-f9bd556f82dc None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:56.244127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e5c8a-acb1-4d35-ab5b-f9bd556f82dc None admin] RBAC: Adding query filter params (type=image) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:56.244403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e5c8a-acb1-4d35-ab5b-f9bd556f82dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:56.245812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e5c8a-acb1-4d35-ab5b-f9bd556f82dc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:56.250596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 498/994] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:05:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:57.973646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65f9ec-c897-4448-b001-ac5e7c3e33e4 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:57.974213 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65f9ec-c897-4448-b001-ac5e7c3e33e4 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:57.974213 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65f9ec-c897-4448-b001-ac5e7c3e33e4 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:57.975556 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 497/995] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:57.981192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760097d1-503c-4c42-a4fa-ec2a94960857 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:57.981665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760097d1-503c-4c42-a4fa-ec2a94960857 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:57.981665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760097d1-503c-4c42-a4fa-ec2a94960857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:58.001973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-760097d1-503c-4c42-a4fa-ec2a94960857 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:58.003158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-760097d1-503c-4c42-a4fa-ec2a94960857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:58.044185 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5135b7-e8b8-469e-a132-7019ff3d474b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:58.044525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5135b7-e8b8-469e-a132-7019ff3d474b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:58.044745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5135b7-e8b8-469e-a132-7019ff3d474b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:58.046342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 498/996] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:05:58.051384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf1d7-4a10-4f70-9476-39c98bece2bd None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:58.051679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf1d7-4a10-4f70-9476-39c98bece2bd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:58.051934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf1d7-4a10-4f70-9476-39c98bece2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:58.070411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-970bf1d7-4a10-4f70-9476-39c98bece2bd None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:58.071815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-970bf1d7-4a10-4f70-9476-39c98bece2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:58.077646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 499/997] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:58.123920 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:58.142583 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 499/998] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:05:58.343579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760097d1-503c-4c42-a4fa-ec2a94960857 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:58.403311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:58.410343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bf1d7-4a10-4f70-9476-39c98bece2bd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:58.449099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760097d1-503c-4c42-a4fa-ec2a94960857 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:58.482805 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8e6ea-4bec-4f69-b82f-b56c3550b53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:58.483435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8e6ea-4bec-4f69-b82f-b56c3550b53f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:58.484005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8e6ea-4bec-4f69-b82f-b56c3550b53f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:58.484273 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8e6ea-4bec-4f69-b82f-b56c3550b53f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:58.484882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8e6ea-4bec-4f69-b82f-b56c3550b53f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:58.486237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8e6ea-4bec-4f69-b82f-b56c3550b53f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-a5c8e6ea-4bec-4f69-b82f-b56c3550b53f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:58.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:58.496207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:05:58.497181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 500/999] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:05:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:05:58.497969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb25f7-c903-4d12-8804-606d50c3fefc None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:58.497969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb25f7-c903-4d12-8804-606d50c3fefc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:58.497969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb25f7-c903-4d12-8804-606d50c3fefc None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:58.499011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 501/1000] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:05:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:05:58.503654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb25f7-c903-4d12-8804-606d50c3fefc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:58.517773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bf1d7-4a10-4f70-9476-39c98bece2bd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:58.552183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ddf21e-48ff-4ef4-9e3d-b3365f8521e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:58.552862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ddf21e-48ff-4ef4-9e3d-b3365f8521e6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:58.553132 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ddf21e-48ff-4ef4-9e3d-b3365f8521e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:58.553395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ddf21e-48ff-4ef4-9e3d-b3365f8521e6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:58.554008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ddf21e-48ff-4ef4-9e3d-b3365f8521e6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:58.555414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ddf21e-48ff-4ef4-9e3d-b3365f8521e6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-19ddf21e-48ff-4ef4-9e3d-b3365f8521e6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:05:58.564276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:05:58.565356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:05:58.570192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 500/1001] 10.222.0.75 () {56 vars in 1126 bytes} [Fri Sep 23 15:05:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:05:58.571790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4255ce-b7c0-4494-b17c-dc53919473d7 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:58.571790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4255ce-b7c0-4494-b17c-dc53919473d7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:58.571790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4255ce-b7c0-4494-b17c-dc53919473d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:58.585583 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ef4255ce-b7c0-4494-b17c-dc53919473d7 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:05:58.586827 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ef4255ce-b7c0-4494-b17c-dc53919473d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:05:58.619153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb25f7-c903-4d12-8804-606d50c3fefc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:58.630700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:58.639318 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:58.654078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13506de0-b8aa-49b5-8d51-526cf5105d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:58.654692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13506de0-b8aa-49b5-8d51-526cf5105d47 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:58.654838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 501/1002] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:05:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:05:58.655025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13506de0-b8aa-49b5-8d51-526cf5105d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:58.655312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13506de0-b8aa-49b5-8d51-526cf5105d47 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:58.655610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4255ce-b7c0-4494-b17c-dc53919473d7 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:58.656021 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13506de0-b8aa-49b5-8d51-526cf5105d47 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:58.656248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13506de0-b8aa-49b5-8d51-526cf5105d47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:58.657683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13506de0-b8aa-49b5-8d51-526cf5105d47 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:58.662665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 502/1003] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Sep 23 15:05:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:58.668411 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13506de0-b8aa-49b5-8d51-526cf5105d47 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:58.766708 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4255ce-b7c0-4494-b17c-dc53919473d7 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:58.779620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13506de0-b8aa-49b5-8d51-526cf5105d47 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:58.802456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b08bc-bebe-4a11-92c2-34e2a8615a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:58.803059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b08bc-bebe-4a11-92c2-34e2a8615a69 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:58.803325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b08bc-bebe-4a11-92c2-34e2a8615a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:58.803588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b08bc-bebe-4a11-92c2-34e2a8615a69 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:58.804207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b08bc-bebe-4a11-92c2-34e2a8615a69 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:58.804502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b08bc-bebe-4a11-92c2-34e2a8615a69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:58.805936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b08bc-bebe-4a11-92c2-34e2a8615a69 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:58.811046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 502/1004] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Sep 23 15:05:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:58.816749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b08bc-bebe-4a11-92c2-34e2a8615a69 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:58.819918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbba1fe-23bb-4d54-8545-6c3e3d26e2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:58.820503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbba1fe-23bb-4d54-8545-6c3e3d26e2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:58.820851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbba1fe-23bb-4d54-8545-6c3e3d26e2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:58.821119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbba1fe-23bb-4d54-8545-6c3e3d26e2e4 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:58.821848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbba1fe-23bb-4d54-8545-6c3e3d26e2e4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:58.822179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbba1fe-23bb-4d54-8545-6c3e3d26e2e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:58.823536 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbba1fe-23bb-4d54-8545-6c3e3d26e2e4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:58.829210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 503/1005] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Sep 23 15:05:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:58.834948 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbba1fe-23bb-4d54-8545-6c3e3d26e2e4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:58.864024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:58.890538 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:58.927633 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b08bc-bebe-4a11-92c2-34e2a8615a69 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:58.943619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbba1fe-23bb-4d54-8545-6c3e3d26e2e4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:58.961551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7738c-3e33-449e-a4af-1dddec7d1cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:58.962223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7738c-3e33-449e-a4af-1dddec7d1cef None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:58.962501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7738c-3e33-449e-a4af-1dddec7d1cef None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:58.962772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7738c-3e33-449e-a4af-1dddec7d1cef None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:58.963406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7738c-3e33-449e-a4af-1dddec7d1cef None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:58.963703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7738c-3e33-449e-a4af-1dddec7d1cef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:58.965349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7738c-3e33-449e-a4af-1dddec7d1cef None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:58.970007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 503/1006] 10.222.0.75 () {56 vars in 1122 bytes} [Fri Sep 23 15:05:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:05:58.970758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7738c-3e33-449e-a4af-1dddec7d1cef None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:58.983341 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9478d31-d0aa-42a6-a1ed-4b9ec8d69128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:58.984190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9478d31-d0aa-42a6-a1ed-4b9ec8d69128 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:58.984190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9478d31-d0aa-42a6-a1ed-4b9ec8d69128 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:58.984443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9478d31-d0aa-42a6-a1ed-4b9ec8d69128 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:58.985187 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9478d31-d0aa-42a6-a1ed-4b9ec8d69128 None admin] RBAC: Adding query filter params (interface=public, service_id=50ca3f78f6bd4b4fae1f4bca1c2ed3bb, region_id=RegionOne) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:05:58.985491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9478d31-d0aa-42a6-a1ed-4b9ec8d69128 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:58.986859 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9478d31-d0aa-42a6-a1ed-4b9ec8d69128 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:58.991813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 504/1007] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Sep 23 15:05:58 2022] GET /identity/v3/endpoints?service_id=50ca3f78f6bd4b4fae1f4bca1c2ed3bb&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:05:58.992629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9478d31-d0aa-42a6-a1ed-4b9ec8d69128 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:59.067596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7738c-3e33-449e-a4af-1dddec7d1cef None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:59.075249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:59.092742 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9478d31-d0aa-42a6-a1ed-4b9ec8d69128 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:59.101971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec3ab1-e2f4-4f65-8f5c-922097bf4f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:59.102623 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec3ab1-e2f4-4f65-8f5c-922097bf4f9f None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:59.102898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec3ab1-e2f4-4f65-8f5c-922097bf4f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:59.103166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec3ab1-e2f4-4f65-8f5c-922097bf4f9f None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:59.103845 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec3ab1-e2f4-4f65-8f5c-922097bf4f9f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:59.105083 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:05:59.105349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec3ab1-e2f4-4f65-8f5c-922097bf4f9f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:59.122711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 504/1008] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:05:58 2022] POST /identity/v3/services => generated 242 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:05:59.123529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec3ab1-e2f4-4f65-8f5c-922097bf4f9f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:05:59.143302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0731c-0963-4c63-8527-8f01ba37ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:59.143811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0731c-0963-4c63-8527-8f01ba37ca52 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:59.144087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0731c-0963-4c63-8527-8f01ba37ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:59.144360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0731c-0963-4c63-8527-8f01ba37ca52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:59.145087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0731c-0963-4c63-8527-8f01ba37ca52 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:59.146494 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0731c-0963-4c63-8527-8f01ba37ca52 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:59.176625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 505/1009] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:05:58 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:05:59.260159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec3ab1-e2f4-4f65-8f5c-922097bf4f9f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:05:59.306597 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6bb53-7f2c-4fa8-9f56-b02b89ea1a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:05:59.307247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6bb53-7f2c-4fa8-9f56-b02b89ea1a91 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:05:59.307594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6bb53-7f2c-4fa8-9f56-b02b89ea1a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:05:59.307865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6bb53-7f2c-4fa8-9f56-b02b89ea1a91 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:05:59.308495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6bb53-7f2c-4fa8-9f56-b02b89ea1a91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:05:59.309936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6bb53-7f2c-4fa8-9f56-b02b89ea1a91 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:05:59.318128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 505/1010] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Sep 23 15:05:59 2022] GET /identity/v3/services => generated 1957 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:06:01.343151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b7e96-876e-40d9-831a-a3d4dff095d9 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:01.343702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b7e96-876e-40d9-831a-a3d4dff095d9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:01.343702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59b7e96-876e-40d9-831a-a3d4dff095d9 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:01.345188 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 506/1011] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:01.351277 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220430f5-13a1-4edd-a8d1-098087f1d53b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:01.351908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220430f5-13a1-4edd-a8d1-098087f1d53b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:01.351908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220430f5-13a1-4edd-a8d1-098087f1d53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:01.373412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-220430f5-13a1-4edd-a8d1-098087f1d53b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:01.374651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-220430f5-13a1-4edd-a8d1-098087f1d53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:01.393268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868333d-995b-4ff4-8a5f-b4b6826c9391 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:01.393511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868333d-995b-4ff4-8a5f-b4b6826c9391 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:01.393790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868333d-995b-4ff4-8a5f-b4b6826c9391 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:01.395326 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 507/1012] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:06:01.404181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:01.404400 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a9c8d-d809-412e-a647-7c22cb5ebd5d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:01.404400 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a9c8d-d809-412e-a647-7c22cb5ebd5d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:01.404400 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a9c8d-d809-412e-a647-7c22cb5ebd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:01.422429 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7c3a9c8d-d809-412e-a647-7c22cb5ebd5d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:01.423620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7c3a9c8d-d809-412e-a647-7c22cb5ebd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:01.456644 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 506/1013] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:01 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:06:01.471585 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d60c3f-38c6-4237-8b1f-7815cb2659de None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:01.471870 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d60c3f-38c6-4237-8b1f-7815cb2659de None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:01.472127 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d60c3f-38c6-4237-8b1f-7815cb2659de None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:01.473630 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 507/1014] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:06:01.478650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61098a7-18d9-40f9-9a9d-1798ced2892f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:01.478927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61098a7-18d9-40f9-9a9d-1798ced2892f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:01.479189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61098a7-18d9-40f9-9a9d-1798ced2892f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:01.498234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e61098a7-18d9-40f9-9a9d-1798ced2892f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:01.499513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e61098a7-18d9-40f9-9a9d-1798ced2892f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:01.504629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 508/1015] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:01 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:01.580683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 508/1016] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:01 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:06:01.716727 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a9c8d-d809-412e-a647-7c22cb5ebd5d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:01.779406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61098a7-18d9-40f9-9a9d-1798ced2892f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:01.819972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:01.825049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a9c8d-d809-412e-a647-7c22cb5ebd5d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:01.861996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd1926-8fd3-4de8-b929-711a6d81eec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:01.862628 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd1926-8fd3-4de8-b929-711a6d81eec6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:01.862901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd1926-8fd3-4de8-b929-711a6d81eec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:01.863165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd1926-8fd3-4de8-b929-711a6d81eec6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:01.864262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd1926-8fd3-4de8-b929-711a6d81eec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:01.865733 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd1926-8fd3-4de8-b929-711a6d81eec6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:01.867340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 509/1017] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:01.868116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd1926-8fd3-4de8-b929-711a6d81eec6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:01.870157 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:01.885998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61098a7-18d9-40f9-9a9d-1798ced2892f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:01.919829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c972c-05b1-48ec-b3b5-fcc643b6ab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:01.920448 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c972c-05b1-48ec-b3b5-fcc643b6ab5b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:01.920745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c972c-05b1-48ec-b3b5-fcc643b6ab5b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:01.921010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c972c-05b1-48ec-b3b5-fcc643b6ab5b None admin] PATH_INFO: `/v3/services/image` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:01.921588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c972c-05b1-48ec-b3b5-fcc643b6ab5b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:01.922936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c972c-05b1-48ec-b3b5-fcc643b6ab5b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-ab3c972c-05b1-48ec-b3b5-fcc643b6ab5b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:01.931910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:01.932991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:01.933835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 509/1018] 10.222.0.75 () {56 vars in 1105 bytes} [Fri Sep 23 15:06:01 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:01.934646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c972c-05b1-48ec-b3b5-fcc643b6ab5b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:01.972931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd1926-8fd3-4de8-b929-711a6d81eec6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:02.009054 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fbbf6-770a-45fa-839f-ab92ebf188c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:02.009679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fbbf6-770a-45fa-839f-ab92ebf188c5 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:02.009971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fbbf6-770a-45fa-839f-ab92ebf188c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:02.010234 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5fbbf6-770a-45fa-839f-ab92ebf188c5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:02.010816 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fbbf6-770a-45fa-839f-ab92ebf188c5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:02.012271 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5fbbf6-770a-45fa-839f-ab92ebf188c5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-0a5fbbf6-770a-45fa-839f-ab92ebf188c5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:02.021062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:02.022066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:02.023349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 510/1019] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:06:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:02.024199 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fbbf6-770a-45fa-839f-ab92ebf188c5 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:02.044085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c972c-05b1-48ec-b3b5-fcc643b6ab5b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:02.056507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:02.082061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6f832-5b88-4c10-ac98-7691b173a5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:02.082667 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6f832-5b88-4c10-ac98-7691b173a5da None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:02.082940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6f832-5b88-4c10-ac98-7691b173a5da None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:02.083208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6f832-5b88-4c10-ac98-7691b173a5da None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:02.083857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6f832-5b88-4c10-ac98-7691b173a5da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:02.085319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6f832-5b88-4c10-ac98-7691b173a5da None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:02.114364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 510/1020] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:06:01 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:06:02.115568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6f832-5b88-4c10-ac98-7691b173a5da None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:02.117923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:02.129989 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5fbbf6-770a-45fa-839f-ab92ebf188c5 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:02.169619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04493c8e-64ba-49aa-b2e5-17cfda3cfe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:02.170366 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04493c8e-64ba-49aa-b2e5-17cfda3cfe38 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:02.170699 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04493c8e-64ba-49aa-b2e5-17cfda3cfe38 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:02.171029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04493c8e-64ba-49aa-b2e5-17cfda3cfe38 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:02.171741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04493c8e-64ba-49aa-b2e5-17cfda3cfe38 None admin] RBAC: Adding query filter params (name=image) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:02.172039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04493c8e-64ba-49aa-b2e5-17cfda3cfe38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:02.173478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04493c8e-64ba-49aa-b2e5-17cfda3cfe38 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:02.184523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 511/1021] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:02.185198 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04493c8e-64ba-49aa-b2e5-17cfda3cfe38 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:02.237579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6f832-5b88-4c10-ac98-7691b173a5da None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:02.281257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7df812-28f0-499c-826a-98d76f478b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:02.281869 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7df812-28f0-499c-826a-98d76f478b16 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:02.282138 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7df812-28f0-499c-826a-98d76f478b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:02.282463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7df812-28f0-499c-826a-98d76f478b16 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:02.283026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7df812-28f0-499c-826a-98d76f478b16 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:02.283459 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7df812-28f0-499c-826a-98d76f478b16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:02.284697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7df812-28f0-499c-826a-98d76f478b16 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:02.290271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 511/1022] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Sep 23 15:06:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:02.293314 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:02.296275 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7df812-28f0-499c-826a-98d76f478b16 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:02.303173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04493c8e-64ba-49aa-b2e5-17cfda3cfe38 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:02.347328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8abcd-35f1-4057-b328-557a2a499e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:02.348034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8abcd-35f1-4057-b328-557a2a499e9b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:02.348260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8abcd-35f1-4057-b328-557a2a499e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:02.348529 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8abcd-35f1-4057-b328-557a2a499e9b None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:02.350721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8abcd-35f1-4057-b328-557a2a499e9b None admin] RBAC: Adding query filter params (type=image) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:02.350721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8abcd-35f1-4057-b328-557a2a499e9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:02.350907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8abcd-35f1-4057-b328-557a2a499e9b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:02.356660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 512/1023] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:02.362997 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8abcd-35f1-4057-b328-557a2a499e9b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:02.375835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:02.413441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7df812-28f0-499c-826a-98d76f478b16 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:02.454595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19e3af-ba85-4561-aae6-1daf4a1b9abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:02.455632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19e3af-ba85-4561-aae6-1daf4a1b9abf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:02.455909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19e3af-ba85-4561-aae6-1daf4a1b9abf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:02.456287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19e3af-ba85-4561-aae6-1daf4a1b9abf None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:02.456831 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19e3af-ba85-4561-aae6-1daf4a1b9abf None admin] RBAC: Adding query filter params (type=compute) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:02.457131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19e3af-ba85-4561-aae6-1daf4a1b9abf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:02.458483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19e3af-ba85-4561-aae6-1daf4a1b9abf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:02.463570 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 512/1024] 10.222.0.75 () {56 vars in 1112 bytes} [Fri Sep 23 15:06:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:02.470239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19e3af-ba85-4561-aae6-1daf4a1b9abf None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:02.472512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8abcd-35f1-4057-b328-557a2a499e9b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:02.509369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738048bc-ca61-43b3-9a01-641056d6f1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:02.509887 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738048bc-ca61-43b3-9a01-641056d6f1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:02.510160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738048bc-ca61-43b3-9a01-641056d6f1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:02.510423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738048bc-ca61-43b3-9a01-641056d6f1a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:02.511127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738048bc-ca61-43b3-9a01-641056d6f1a8 None admin] RBAC: Adding query filter params (interface=public, service_id=8dc59e24dd084958899d4c736658b382, region_id=RegionOne) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:02.511427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738048bc-ca61-43b3-9a01-641056d6f1a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:02.512814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738048bc-ca61-43b3-9a01-641056d6f1a8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:02.518073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 513/1025] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Sep 23 15:06:02 2022] GET /identity/v3/endpoints?service_id=8dc59e24dd084958899d4c736658b382&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:02.523682 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738048bc-ca61-43b3-9a01-641056d6f1a8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:02.559975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:02.589323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19e3af-ba85-4561-aae6-1daf4a1b9abf None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:02.618745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:02.625240 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999c5c0-0eb8-4440-a784-c4b143cef743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:02.625850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999c5c0-0eb8-4440-a784-c4b143cef743 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:02.626121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999c5c0-0eb8-4440-a784-c4b143cef743 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:02.626380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999c5c0-0eb8-4440-a784-c4b143cef743 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:02.627126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999c5c0-0eb8-4440-a784-c4b143cef743 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:02.628416 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999c5c0-0eb8-4440-a784-c4b143cef743 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:02.647283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738048bc-ca61-43b3-9a01-641056d6f1a8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:02.660618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 513/1026] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:06:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:06:02.695234 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5997c780-1c42-4cc8-ba5b-24fd0161cb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:02.696013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997c780-1c42-4cc8-ba5b-24fd0161cb2e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:02.696514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997c780-1c42-4cc8-ba5b-24fd0161cb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:02.696882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5997c780-1c42-4cc8-ba5b-24fd0161cb2e None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:02.697518 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997c780-1c42-4cc8-ba5b-24fd0161cb2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:02.698949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5997c780-1c42-4cc8-ba5b-24fd0161cb2e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:02.706550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 514/1027] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Sep 23 15:06:02 2022] GET /identity/v3/services => generated 1957 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:06:04.325867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34bb64-ba59-4f87-bc8c-ccb71e128930 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:04.326456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34bb64-ba59-4f87-bc8c-ccb71e128930 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:04.326456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34bb64-ba59-4f87-bc8c-ccb71e128930 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:04.327915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 514/1028] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:04.333244 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e559faa-0524-4ec9-82f8-868e691681ba None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:04.333669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e559faa-0524-4ec9-82f8-868e691681ba None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:04.333743 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e559faa-0524-4ec9-82f8-868e691681ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:04.352425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4e559faa-0524-4ec9-82f8-868e691681ba None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:04.353640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4e559faa-0524-4ec9-82f8-868e691681ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:04.420762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 515/1029] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:04.685298 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34bb64-ba59-4f87-bc8c-ccb71e128930 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:04.807723 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34bb64-ba59-4f87-bc8c-ccb71e128930 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:04.850470 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a235d86-1df6-4a55-b522-fb11b2255a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:04.851126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a235d86-1df6-4a55-b522-fb11b2255a46 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:04.851388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a235d86-1df6-4a55-b522-fb11b2255a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:04.851643 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a235d86-1df6-4a55-b522-fb11b2255a46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:04.853038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a235d86-1df6-4a55-b522-fb11b2255a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:04.854515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a235d86-1df6-4a55-b522-fb11b2255a46 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:04.856520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 515/1030] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:04.864636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e559faa-0524-4ec9-82f8-868e691681ba None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:04.885139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:04.919802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a4039-2d03-4da4-b1d9-4a76e63757ae None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:04.920164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a4039-2d03-4da4-b1d9-4a76e63757ae None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:04.920310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a4039-2d03-4da4-b1d9-4a76e63757ae None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:04.921786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 516/1031] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:04.927135 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadfcc4-7a8d-4bdf-824a-80fdac1562c6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:04.927414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadfcc4-7a8d-4bdf-824a-80fdac1562c6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:04.927714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadfcc4-7a8d-4bdf-824a-80fdac1562c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:04.947288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4fadfcc4-7a8d-4bdf-824a-80fdac1562c6 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:04.948385 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4fadfcc4-7a8d-4bdf-824a-80fdac1562c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:04.992944 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e559faa-0524-4ec9-82f8-868e691681ba None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:05.009838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:05.030899 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 517/1032] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:06:05.031950 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d9657-85d6-4186-9b00-ede8835a17b5 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:05.032259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d9657-85d6-4186-9b00-ede8835a17b5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:05.032552 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d9657-85d6-4186-9b00-ede8835a17b5 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:05.034017 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 518/1033] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:05.037328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d79d2-a40a-49c7-852a-ef3b1c6b8708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:05.038134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d79d2-a40a-49c7-852a-ef3b1c6b8708 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:05.038480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d79d2-a40a-49c7-852a-ef3b1c6b8708 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:05.038832 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d79d2-a40a-49c7-852a-ef3b1c6b8708 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:05.039752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d79d2-a40a-49c7-852a-ef3b1c6b8708 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:05.041280 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d79d2-a40a-49c7-852a-ef3b1c6b8708 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:05.042586 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768e45f-7976-4b5c-be3a-da3b253bbd57 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:05.042938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768e45f-7976-4b5c-be3a-da3b253bbd57 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:05.043212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768e45f-7976-4b5c-be3a-da3b253bbd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:05.065449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4768e45f-7976-4b5c-be3a-da3b253bbd57 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:05.066860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4768e45f-7976-4b5c-be3a-da3b253bbd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:05.068677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 516/1034] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:06:04 2022] POST /identity/v3/users => generated 285 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:06:05.164214 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 519/1035] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:05.321509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d79d2-a40a-49c7-852a-ef3b1c6b8708 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:05.402962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:05.427922 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768e45f-7976-4b5c-be3a-da3b253bbd57 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:05.449224 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d79d2-a40a-49c7-852a-ef3b1c6b8708 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:05.491723 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd26f1-69e2-4abb-9cbc-011dd75740e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:05.492367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd26f1-69e2-4abb-9cbc-011dd75740e1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:05.492927 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd26f1-69e2-4abb-9cbc-011dd75740e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:05.493397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd26f1-69e2-4abb-9cbc-011dd75740e1 None admin] PATH_INFO: `/v3/services/image` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:05.494149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd26f1-69e2-4abb-9cbc-011dd75740e1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:05.496374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd26f1-69e2-4abb-9cbc-011dd75740e1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-1ebd26f1-69e2-4abb-9cbc-011dd75740e1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:05.508171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:05.509406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:05.510267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 517/1036] 10.222.0.75 () {56 vars in 1105 bytes} [Fri Sep 23 15:06:05 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:05.515613 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd26f1-69e2-4abb-9cbc-011dd75740e1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:05.544323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768e45f-7976-4b5c-be3a-da3b253bbd57 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:05.562253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:05.587369 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1c34d-16e7-4984-85b9-6636b9db67d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:05.587923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1c34d-16e7-4984-85b9-6636b9db67d5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:05.588266 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1c34d-16e7-4984-85b9-6636b9db67d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:05.588621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1c34d-16e7-4984-85b9-6636b9db67d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:05.601532 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1c34d-16e7-4984-85b9-6636b9db67d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-4fc1c34d-16e7-4984-85b9-6636b9db67d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:06:05.607898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:05.609239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:05.610085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:05.610085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:05.610085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:05.610085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:05.610085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:06:05.610085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:06:05.610085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:06:05.610085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:06:05.610085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:06:05.610085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:06:05.610085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:06:05.610085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:05.610085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 520/1037] 10.222.0.75 () {56 vars in 1120 bytes} [Fri Sep 23 15:06:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:06:05.615774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1c34d-16e7-4984-85b9-6636b9db67d5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:05.632444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd26f1-69e2-4abb-9cbc-011dd75740e1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:05.663539 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:05.670989 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1b785-c3a5-4618-8966-3a51c0f424ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:05.672030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1b785-c3a5-4618-8966-3a51c0f424ef None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:05.672303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1b785-c3a5-4618-8966-3a51c0f424ef None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:05.672743 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1b785-c3a5-4618-8966-3a51c0f424ef None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:05.673410 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1b785-c3a5-4618-8966-3a51c0f424ef None admin] RBAC: Adding query filter params (name=image) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:05.673713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1b785-c3a5-4618-8966-3a51c0f424ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:05.675101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1b785-c3a5-4618-8966-3a51c0f424ef None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:05.684634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 518/1038] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:05.686766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1b785-c3a5-4618-8966-3a51c0f424ef None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:05.742735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1c34d-16e7-4984-85b9-6636b9db67d5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:05.784979 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037bea9-0301-4133-b470-448ea3d3a482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:05.786541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037bea9-0301-4133-b470-448ea3d3a482 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:05.787020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037bea9-0301-4133-b470-448ea3d3a482 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:05.787573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037bea9-0301-4133-b470-448ea3d3a482 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:05.788530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037bea9-0301-4133-b470-448ea3d3a482 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:05.789038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037bea9-0301-4133-b470-448ea3d3a482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:05.790867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037bea9-0301-4133-b470-448ea3d3a482 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:05.794500 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1b785-c3a5-4618-8966-3a51c0f424ef None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:05.803882 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 521/1039] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Sep 23 15:06:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:05.810350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037bea9-0301-4133-b470-448ea3d3a482 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:05.833722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6e978-2960-4213-9344-9a469e77f48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:05.835167 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6e978-2960-4213-9344-9a469e77f48b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:05.835663 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6e978-2960-4213-9344-9a469e77f48b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:05.836348 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6e978-2960-4213-9344-9a469e77f48b None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:05.837340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6e978-2960-4213-9344-9a469e77f48b None admin] RBAC: Adding query filter params (type=image) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:05.837889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6e978-2960-4213-9344-9a469e77f48b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:05.840216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6e978-2960-4213-9344-9a469e77f48b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:05.847441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 519/1040] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:05.853829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6e978-2960-4213-9344-9a469e77f48b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:05.854662 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:05.921285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037bea9-0301-4133-b470-448ea3d3a482 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:05.923903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:05.958116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c288457-ae35-4a43-b3c1-8ede73349bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:05.958744 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c288457-ae35-4a43-b3c1-8ede73349bcc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:05.959121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c288457-ae35-4a43-b3c1-8ede73349bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:05.959386 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c288457-ae35-4a43-b3c1-8ede73349bcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:05.966264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6e978-2960-4213-9344-9a469e77f48b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:05.970647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c288457-ae35-4a43-b3c1-8ede73349bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:05.972084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c288457-ae35-4a43-b3c1-8ede73349bcc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-2c288457-ae35-4a43-b3c1-8ede73349bcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:06:05.987721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:05.988737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:06:05.990049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:06:05.990049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:05.990049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:05.990049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 522/1041] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:05.995050 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c288457-ae35-4a43-b3c1-8ede73349bcc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:06.011453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e9fad-a04a-406c-9a66-98986999d4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:06.012073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e9fad-a04a-406c-9a66-98986999d4d4 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:06.012347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e9fad-a04a-406c-9a66-98986999d4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:06.012645 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e9fad-a04a-406c-9a66-98986999d4d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:06.013342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e9fad-a04a-406c-9a66-98986999d4d4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:06.014720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e9fad-a04a-406c-9a66-98986999d4d4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:06.038876 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 520/1042] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:06:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:06:06.114129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:06.130740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c288457-ae35-4a43-b3c1-8ede73349bcc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:06.178527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9fef2-7306-47bd-9753-3efb4154bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:06.179149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9fef2-7306-47bd-9753-3efb4154bf11 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:06.179417 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9fef2-7306-47bd-9753-3efb4154bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:06.179679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9fef2-7306-47bd-9753-3efb4154bf11 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:06.180753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9fef2-7306-47bd-9753-3efb4154bf11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:06.181168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9fef2-7306-47bd-9753-3efb4154bf11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:06.182575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9fef2-7306-47bd-9753-3efb4154bf11 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:06.200798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 523/1043] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:06.203622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e9fad-a04a-406c-9a66-98986999d4d4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:06.318672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e9fad-a04a-406c-9a66-98986999d4d4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:06.336453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:06.357480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a332c-77b4-4002-be74-b670a875ba1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:06.358114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a332c-77b4-4002-be74-b670a875ba1f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:06.358384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a332c-77b4-4002-be74-b670a875ba1f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:06.358955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a332c-77b4-4002-be74-b670a875ba1f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:06.367437 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a332c-77b4-4002-be74-b670a875ba1f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:06.368857 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a332c-77b4-4002-be74-b670a875ba1f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-762a332c-77b4-4002-be74-b670a875ba1f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:06:06.380485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:06.381499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:06.382485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:06.382485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:06.382485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:06:06.382485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:06:06.382485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:06:06.382485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:06:06.382485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:06:06.382485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:06:06.382485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:06:06.382485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:06.382485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 521/1044] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Sep 23 15:06:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:06.387620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9fef2-7306-47bd-9753-3efb4154bf11 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:06.491506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9fef2-7306-47bd-9753-3efb4154bf11 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:06.531032 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca740c0-b581-45e0-91eb-7ba222a339b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:06.531645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca740c0-b581-45e0-91eb-7ba222a339b5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:06.531915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca740c0-b581-45e0-91eb-7ba222a339b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:06.532177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca740c0-b581-45e0-91eb-7ba222a339b5 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:06.532956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca740c0-b581-45e0-91eb-7ba222a339b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:06.536118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca740c0-b581-45e0-91eb-7ba222a339b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:06.536118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca740c0-b581-45e0-91eb-7ba222a339b5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:06.556478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 524/1045] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Sep 23 15:06:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:06.562867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a332c-77b4-4002-be74-b670a875ba1f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:06.670544 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a332c-77b4-4002-be74-b670a875ba1f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:06.704445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc6b29-5894-4809-af04-a8c75f52e09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:06.705075 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc6b29-5894-4809-af04-a8c75f52e09d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:06.705352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc6b29-5894-4809-af04-a8c75f52e09d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:06.705615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc6b29-5894-4809-af04-a8c75f52e09d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:06.751492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:06.777148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc6b29-5894-4809-af04-a8c75f52e09d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:06.778597 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc6b29-5894-4809-af04-a8c75f52e09d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:06.880369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 522/1046] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:06:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:06:06.886425 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca740c0-b581-45e0-91eb-7ba222a339b5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:06.889737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:06.989777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca740c0-b581-45e0-91eb-7ba222a339b5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:07.028125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7631d-62b2-4a23-b4a0-1262fae19121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:07.028792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7631d-62b2-4a23-b4a0-1262fae19121 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:07.029058 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7631d-62b2-4a23-b4a0-1262fae19121 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:07.029322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7631d-62b2-4a23-b4a0-1262fae19121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:07.039689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7631d-62b2-4a23-b4a0-1262fae19121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:07.041293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7631d-62b2-4a23-b4a0-1262fae19121 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-83f7631d-62b2-4a23-b4a0-1262fae19121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:06:07.056184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:07.057384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:06:07.058377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:06:07.058377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:07.058377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:07.058377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 525/1047] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:07.064529 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc6b29-5894-4809-af04-a8c75f52e09d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:07.147466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:07.174342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc6b29-5894-4809-af04-a8c75f52e09d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:07.209385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e73288-6830-4c72-8534-607e453723d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:07.210006 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e73288-6830-4c72-8534-607e453723d6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:07.210280 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e73288-6830-4c72-8534-607e453723d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:07.210551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e73288-6830-4c72-8534-607e453723d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:07.211237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e73288-6830-4c72-8534-607e453723d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:07.211536 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e73288-6830-4c72-8534-607e453723d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:07.212988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e73288-6830-4c72-8534-607e453723d6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:07.226511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 523/1048] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:07.232623 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7631d-62b2-4a23-b4a0-1262fae19121 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:07.294370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:07.341270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7631d-62b2-4a23-b4a0-1262fae19121 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:07.341571 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14907cb-198c-422a-baef-4aec2359ac64 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:07.341853 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14907cb-198c-422a-baef-4aec2359ac64 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:07.342120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14907cb-198c-422a-baef-4aec2359ac64 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:07.343581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 524/1049] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:06:07.348714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2eb9b-f1e3-4a5b-9f91-934262ccf513 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:07.349005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2eb9b-f1e3-4a5b-9f91-934262ccf513 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:07.349271 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2eb9b-f1e3-4a5b-9f91-934262ccf513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:07.367981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3fd2eb9b-f1e3-4a5b-9f91-934262ccf513 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:07.369319 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3fd2eb9b-f1e3-4a5b-9f91-934262ccf513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:07.382285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca773cad-86b9-439a-a92d-b888a5e37665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:07.382918 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca773cad-86b9-439a-a92d-b888a5e37665 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:07.383268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca773cad-86b9-439a-a92d-b888a5e37665 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:07.383537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca773cad-86b9-439a-a92d-b888a5e37665 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:07.390493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca773cad-86b9-439a-a92d-b888a5e37665 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:07.391922 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca773cad-86b9-439a-a92d-b888a5e37665 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-ca773cad-86b9-439a-a92d-b888a5e37665 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:06:07.403120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:07.404034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:07.404961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:06:07.404961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:06:07.404961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:06:07.404961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:06:07.404961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:06:07.404961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:06:07.404961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:06:07.404961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:07.404961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 526/1050] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:06:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:07.410264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca773cad-86b9-439a-a92d-b888a5e37665 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:07.449236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 525/1051] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:06:07.514031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca773cad-86b9-439a-a92d-b888a5e37665 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:07.528846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:07.554674 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8f516-e653-48a8-bf8e-fcad7d23a60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:07.555379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8f516-e653-48a8-bf8e-fcad7d23a60e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:07.555652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8f516-e653-48a8-bf8e-fcad7d23a60e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:07.555911 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8f516-e653-48a8-bf8e-fcad7d23a60e None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:07.556624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe8f516-e653-48a8-bf8e-fcad7d23a60e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:07.556911 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe8f516-e653-48a8-bf8e-fcad7d23a60e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:07.558319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe8f516-e653-48a8-bf8e-fcad7d23a60e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:07.571617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 527/1052] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:06:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:07.577678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2eb9b-f1e3-4a5b-9f91-934262ccf513 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:07.645436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:07.683165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2eb9b-f1e3-4a5b-9f91-934262ccf513 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:07.715938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8f516-e653-48a8-bf8e-fcad7d23a60e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:07.721123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1255249-9e36-4e42-89b3-4bdfd1011a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:07.721777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1255249-9e36-4e42-89b3-4bdfd1011a2e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:07.722051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1255249-9e36-4e42-89b3-4bdfd1011a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:07.722376 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1255249-9e36-4e42-89b3-4bdfd1011a2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:07.723138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1255249-9e36-4e42-89b3-4bdfd1011a2e None admin] RBAC: Adding query filter params (role.id=ea4eebb7bada4fe4a89fa7f71f4b09ce, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=2af1ba390fff41a68ff1a0035901c318) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:07.723443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1255249-9e36-4e42-89b3-4bdfd1011a2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:07.725311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1255249-9e36-4e42-89b3-4bdfd1011a2e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:07.732637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 526/1053] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:06:07 2022] GET /identity/v3/role_assignments?user.id=2af1ba390fff41a68ff1a0035901c318&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=ea4eebb7bada4fe4a89fa7f71f4b09ce => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:07.845126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8f516-e653-48a8-bf8e-fcad7d23a60e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:07.884914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e787d0-ce9f-4b09-a2e3-6de50ceb5693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:07.885574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e787d0-ce9f-4b09-a2e3-6de50ceb5693 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:07.885837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e787d0-ce9f-4b09-a2e3-6de50ceb5693 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:07.886098 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e787d0-ce9f-4b09-a2e3-6de50ceb5693 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:07.898034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e787d0-ce9f-4b09-a2e3-6de50ceb5693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-55e787d0-ce9f-4b09-a2e3-6de50ceb5693 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:06:07.903406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:07.904435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:07.909604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:07.909604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:07.909604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:07.909604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:07.909604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:06:07.909604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:06:07.909604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:06:07.909604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:06:07.909604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:06:07.909604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:06:07.909604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:06:07.909604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:07.909604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 528/1054] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:06:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:06:07.911947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1255249-9e36-4e42-89b3-4bdfd1011a2e None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:08.017207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1255249-9e36-4e42-89b3-4bdfd1011a2e None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:08.045838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:08.053547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d0432-975e-49a3-b178-4d6099b01ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:08.054159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d0432-975e-49a3-b178-4d6099b01ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:08.054438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d0432-975e-49a3-b178-4d6099b01ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:08.054705 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d0432-975e-49a3-b178-4d6099b01ba2 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:08.055400 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d0432-975e-49a3-b178-4d6099b01ba2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:08.055688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d0432-975e-49a3-b178-4d6099b01ba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:08.057088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d0432-975e-49a3-b178-4d6099b01ba2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:08.072547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 527/1055] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:06:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:08.074029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e787d0-ce9f-4b09-a2e3-6de50ceb5693 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:08.110259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:08.188029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e787d0-ce9f-4b09-a2e3-6de50ceb5693 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:08.242792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5538bb9-6021-4fc8-9d3c-14009dae8edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:08.243437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5538bb9-6021-4fc8-9d3c-14009dae8edc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:08.243689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5538bb9-6021-4fc8-9d3c-14009dae8edc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:08.243963 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5538bb9-6021-4fc8-9d3c-14009dae8edc None admin] PATH_INFO: `/v3/users/e0b2280d4e72447d9efc4a3ee38628fe` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:08.258560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5538bb9-6021-4fc8-9d3c-14009dae8edc None admin] RBAC: Authorizing `identity:get_user(user_id=e0b2280d4e72447d9efc4a3ee38628fe)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:08.260235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5538bb9-6021-4fc8-9d3c-14009dae8edc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:08.262119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 529/1056] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:06:08 2022] GET /identity/v3/users/e0b2280d4e72447d9efc4a3ee38628fe => generated 285 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:08.267781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d0432-975e-49a3-b178-4d6099b01ba2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:08.278827 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49dbfd-c6d1-4d09-985b-539133220df8 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:08.279123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49dbfd-c6d1-4d09-985b-539133220df8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:08.279745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf49dbfd-c6d1-4d09-985b-539133220df8 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:08.281484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 530/1057] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:08.286503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb9b68-5752-44fa-b23d-cdc4cc340a99 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:08.286775 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb9b68-5752-44fa-b23d-cdc4cc340a99 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:08.287039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb9b68-5752-44fa-b23d-cdc4cc340a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:08.305511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c2fb9b68-5752-44fa-b23d-cdc4cc340a99 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:08.306850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c2fb9b68-5752-44fa-b23d-cdc4cc340a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:08.376384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d0432-975e-49a3-b178-4d6099b01ba2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:08.380642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 531/1058] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:06:08.411680 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c717ba4-5752-4bfd-aeb0-e686beb76a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:08.412279 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c717ba4-5752-4bfd-aeb0-e686beb76a64 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:08.412543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c717ba4-5752-4bfd-aeb0-e686beb76a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:08.412830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c717ba4-5752-4bfd-aeb0-e686beb76a64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:08.473596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:08.485475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c717ba4-5752-4bfd-aeb0-e686beb76a64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:08.487105 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c717ba4-5752-4bfd-aeb0-e686beb76a64 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:08.595938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 528/1059] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:06:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 23 15:06:08.601946 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb9b68-5752-44fa-b23d-cdc4cc340a99 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:08.628807 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:08.645784 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c717ba4-5752-4bfd-aeb0-e686beb76a64 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:08.711274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb9b68-5752-44fa-b23d-cdc4cc340a99 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:08.742222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:08.746580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984fa7f-e0ef-4701-bdde-29116b597715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:08.747224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984fa7f-e0ef-4701-bdde-29116b597715 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:08.747495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984fa7f-e0ef-4701-bdde-29116b597715 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:08.747758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984fa7f-e0ef-4701-bdde-29116b597715 None admin] PATH_INFO: `/v3/projects/638cc303fbcb464d81a07c7cbf57c2da` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:08.748904 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984fa7f-e0ef-4701-bdde-29116b597715 None admin] RBAC: Authorizing `identity:get_project(project_id=638cc303fbcb464d81a07c7cbf57c2da)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:08.750421 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984fa7f-e0ef-4701-bdde-29116b597715 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:08.752080 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 532/1060] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:06:08 2022] GET /identity/v3/projects/638cc303fbcb464d81a07c7cbf57c2da => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:08.758250 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984fa7f-e0ef-4701-bdde-29116b597715 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:08.760258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c717ba4-5752-4bfd-aeb0-e686beb76a64 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:08.795206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae7b44-e97e-4e93-8efe-481e87a4ffcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:08.796088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae7b44-e97e-4e93-8efe-481e87a4ffcb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:08.796374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae7b44-e97e-4e93-8efe-481e87a4ffcb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:08.796647 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae7b44-e97e-4e93-8efe-481e87a4ffcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:08.848832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:08.859326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984fa7f-e0ef-4701-bdde-29116b597715 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:08.873711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae7b44-e97e-4e93-8efe-481e87a4ffcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:08.875158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae7b44-e97e-4e93-8efe-481e87a4ffcb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:08.897934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5ee95-25dc-43ea-9caf-247658d15d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:08.898811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5ee95-25dc-43ea-9caf-247658d15d5e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:08.899398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5ee95-25dc-43ea-9caf-247658d15d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:08.899792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5ee95-25dc-43ea-9caf-247658d15d5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:08.900652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5ee95-25dc-43ea-9caf-247658d15d5e None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=638cc303fbcb464d81a07c7cbf57c2da, user.id=e0b2280d4e72447d9efc4a3ee38628fe) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:08.900954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5ee95-25dc-43ea-9caf-247658d15d5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:08.902381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5ee95-25dc-43ea-9caf-247658d15d5e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:08.909408 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 533/1061] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:06:08 2022] GET /identity/v3/role_assignments?user.id=e0b2280d4e72447d9efc4a3ee38628fe&scope.project.id=638cc303fbcb464d81a07c7cbf57c2da&role.id=ae444a489623434999d758b71ae2dc87 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:08.988979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:09.001888 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 529/1062] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:06:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:06:09.007882 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5ee95-25dc-43ea-9caf-247658d15d5e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:09.115976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5ee95-25dc-43ea-9caf-247658d15d5e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:09.150732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc09478-6221-478d-b309-4f481824cbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:09.151258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09478-6221-478d-b309-4f481824cbd9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:09.151536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09478-6221-478d-b309-4f481824cbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:09.151799 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09478-6221-478d-b309-4f481824cbd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:09.162017 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc09478-6221-478d-b309-4f481824cbd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:09.163447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc09478-6221-478d-b309-4f481824cbd9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-bbc09478-6221-478d-b309-4f481824cbd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:06:09.177925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:09.178881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:06:09.179764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:06:09.179764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:09.179764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:09.180548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 534/1063] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:09.185152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae7b44-e97e-4e93-8efe-481e87a4ffcb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:09.292737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae7b44-e97e-4e93-8efe-481e87a4ffcb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:09.331281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a58b3-6a43-4cad-af42-5a18eda4b06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:09.331887 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a58b3-6a43-4cad-af42-5a18eda4b06a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:09.332169 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a58b3-6a43-4cad-af42-5a18eda4b06a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:09.332413 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a58b3-6a43-4cad-af42-5a18eda4b06a None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:09.333100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a58b3-6a43-4cad-af42-5a18eda4b06a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:09.333390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a58b3-6a43-4cad-af42-5a18eda4b06a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:09.334754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a58b3-6a43-4cad-af42-5a18eda4b06a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:09.346858 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 530/1064] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:09.885371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfeb88-c790-44c7-9e8a-97dfa258dac5 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:09.885929 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfeb88-c790-44c7-9e8a-97dfa258dac5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:09.885929 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfeb88-c790-44c7-9e8a-97dfa258dac5 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:09.887236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 535/1065] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:06:09.892447 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ef082-0cfd-437a-8d11-2b0b2014ca7e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:09.892893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ef082-0cfd-437a-8d11-2b0b2014ca7e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:09.892978 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ef082-0cfd-437a-8d11-2b0b2014ca7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:09.911416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a39ef082-0cfd-437a-8d11-2b0b2014ca7e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:09.912615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a39ef082-0cfd-437a-8d11-2b0b2014ca7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:09.962412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:09.982943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 531/1066] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:10.250633 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfeb88-c790-44c7-9e8a-97dfa258dac5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:10.260800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:10.353594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfeb88-c790-44c7-9e8a-97dfa258dac5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:10.387013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a845a7-0e6b-41aa-bfb0-b6138ce35cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:10.387622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a845a7-0e6b-41aa-bfb0-b6138ce35cec None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:10.387891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a845a7-0e6b-41aa-bfb0-b6138ce35cec None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:10.388133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a845a7-0e6b-41aa-bfb0-b6138ce35cec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:10.397991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a845a7-0e6b-41aa-bfb0-b6138ce35cec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-11a845a7-0e6b-41aa-bfb0-b6138ce35cec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:06:10.402767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:10.403810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:10.404691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:10.404691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:10.404691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:10.404691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:10.404691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:06:10.404691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:06:10.404691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:06:10.404691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:06:10.404691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:06:10.404691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:06:10.404691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:06:10.404691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:10.404691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 536/1067] 10.222.0.75 () {56 vars in 1120 bytes} [Fri Sep 23 15:06:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:10.410079 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ef082-0cfd-437a-8d11-2b0b2014ca7e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:10.512709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ef082-0cfd-437a-8d11-2b0b2014ca7e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:10.551580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dae07-9055-4d5f-a432-bd8824f1b519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:10.552184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dae07-9055-4d5f-a432-bd8824f1b519 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:10.552450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dae07-9055-4d5f-a432-bd8824f1b519 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:10.552740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dae07-9055-4d5f-a432-bd8824f1b519 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:10.553403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5dae07-9055-4d5f-a432-bd8824f1b519 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:10.553693 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5dae07-9055-4d5f-a432-bd8824f1b519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:10.555506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5dae07-9055-4d5f-a432-bd8824f1b519 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:10.567031 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 532/1068] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Sep 23 15:06:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:10.572820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a845a7-0e6b-41aa-bfb0-b6138ce35cec None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:10.641613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:10.676918 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a845a7-0e6b-41aa-bfb0-b6138ce35cec None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:10.714503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d602c9-eb35-462a-a9db-30b9f6efc0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:10.715147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d602c9-eb35-462a-a9db-30b9f6efc0fb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:10.715414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d602c9-eb35-462a-a9db-30b9f6efc0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:10.715669 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d602c9-eb35-462a-a9db-30b9f6efc0fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:10.726867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d602c9-eb35-462a-a9db-30b9f6efc0fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:10.728322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d602c9-eb35-462a-a9db-30b9f6efc0fb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-05d602c9-eb35-462a-a9db-30b9f6efc0fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:06:10.743686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:10.744792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:06:10.745792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:06:10.745792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:10.745792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:10.745792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 537/1069] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:10.750856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dae07-9055-4d5f-a432-bd8824f1b519 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:10.801759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:10.857916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dae07-9055-4d5f-a432-bd8824f1b519 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:10.892603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22cb20-f487-478e-956b-5f93d2b22f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:10.893277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22cb20-f487-478e-956b-5f93d2b22f2d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:10.893435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22cb20-f487-478e-956b-5f93d2b22f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:10.893714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22cb20-f487-478e-956b-5f93d2b22f2d None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:10.894345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22cb20-f487-478e-956b-5f93d2b22f2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:10.894635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22cb20-f487-478e-956b-5f93d2b22f2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:10.895993 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22cb20-f487-478e-956b-5f93d2b22f2d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:10.907787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 533/1070] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:10.913502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d602c9-eb35-462a-a9db-30b9f6efc0fb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:11.023530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d602c9-eb35-462a-a9db-30b9f6efc0fb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:11.041237 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:11.055389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28442145-f06b-4d60-a1cc-62196a03e6f5 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:11.055874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28442145-f06b-4d60-a1cc-62196a03e6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:11.056067 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28442145-f06b-4d60-a1cc-62196a03e6f5 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:11.057701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 534/1071] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:11.059041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd2b42-ca95-4fcc-9d04-2641ba1d7a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:11.059544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd2b42-ca95-4fcc-9d04-2641ba1d7a1f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:11.059651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd2b42-ca95-4fcc-9d04-2641ba1d7a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:11.059929 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd2b42-ca95-4fcc-9d04-2641ba1d7a1f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:11.062800 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d443b-58eb-4739-bdbb-da6301d8d98e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:11.063084 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d443b-58eb-4739-bdbb-da6301d8d98e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:11.063354 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d443b-58eb-4739-bdbb-da6301d8d98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:11.067865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd2b42-ca95-4fcc-9d04-2641ba1d7a1f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:11.069314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd2b42-ca95-4fcc-9d04-2641ba1d7a1f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-94dd2b42-ca95-4fcc-9d04-2641ba1d7a1f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:06:11.080926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:11.082121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:11.083535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:11.083535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:11.083535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:06:11.083535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:06:11.083535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:06:11.083535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:06:11.083535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:06:11.083535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:06:11.083535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:06:11.083535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:11.084111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-480d443b-58eb-4739-bdbb-da6301d8d98e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:11.084190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 538/1072] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Sep 23 15:06:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:11.084263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-480d443b-58eb-4739-bdbb-da6301d8d98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:11.088086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd2b42-ca95-4fcc-9d04-2641ba1d7a1f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:11.151416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 535/1073] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:06:11.188600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd2b42-ca95-4fcc-9d04-2641ba1d7a1f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:11.222995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89ab41-9917-4830-a730-785e9014df32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:11.223587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89ab41-9917-4830-a730-785e9014df32 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:11.223859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89ab41-9917-4830-a730-785e9014df32 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:11.224126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89ab41-9917-4830-a730-785e9014df32 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:11.224905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89ab41-9917-4830-a730-785e9014df32 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:11.225203 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89ab41-9917-4830-a730-785e9014df32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:11.226652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89ab41-9917-4830-a730-785e9014df32 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:11.244300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 539/1074] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Sep 23 15:06:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:11.250278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d443b-58eb-4739-bdbb-da6301d8d98e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:11.284412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:11.351990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d443b-58eb-4739-bdbb-da6301d8d98e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:11.386552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3052f9-f7ef-4b0d-86f1-0e36fa972b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:11.387176 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3052f9-f7ef-4b0d-86f1-0e36fa972b85 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:11.387443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3052f9-f7ef-4b0d-86f1-0e36fa972b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:11.387701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3052f9-f7ef-4b0d-86f1-0e36fa972b85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:11.397712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3052f9-f7ef-4b0d-86f1-0e36fa972b85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:11.399402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3052f9-f7ef-4b0d-86f1-0e36fa972b85 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-1e3052f9-f7ef-4b0d-86f1-0e36fa972b85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:06:11.413757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:11.414904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:06:11.416078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:06:11.416078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:11.416078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:11.416294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89ab41-9917-4830-a730-785e9014df32 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:11.416388 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 536/1075] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:11.422379 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3052f9-f7ef-4b0d-86f1-0e36fa972b85 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:11.456376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:11.513424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:11.525296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89ab41-9917-4830-a730-785e9014df32 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:11.529549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3052f9-f7ef-4b0d-86f1-0e36fa972b85 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:11.563377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76004a3d-bbd0-414f-ade6-c1bea6ddc174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:11.564013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76004a3d-bbd0-414f-ade6-c1bea6ddc174 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:11.564291 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76004a3d-bbd0-414f-ade6-c1bea6ddc174 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:11.564561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76004a3d-bbd0-414f-ade6-c1bea6ddc174 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:11.567641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c8c88-cd2c-4898-a090-78fb7a311b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:11.568249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8c88-cd2c-4898-a090-78fb7a311b9f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:11.568523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8c88-cd2c-4898-a090-78fb7a311b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:11.568811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c8c88-cd2c-4898-a090-78fb7a311b9f None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:11.569465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c8c88-cd2c-4898-a090-78fb7a311b9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:11.569750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c8c88-cd2c-4898-a090-78fb7a311b9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:11.571376 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c8c88-cd2c-4898-a090-78fb7a311b9f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:11.575596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76004a3d-bbd0-414f-ade6-c1bea6ddc174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-76004a3d-bbd0-414f-ade6-c1bea6ddc174 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:06:11.580389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:11.581354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:11.582215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:11.582215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:11.582215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:11.582215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:11.582215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:06:11.582215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:06:11.582215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:06:11.582215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:06:11.582215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:06:11.582215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:06:11.582215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:06:11.582215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:11.582215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 540/1076] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:06:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:11.582960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 537/1077] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:11.588622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c8c88-cd2c-4898-a090-78fb7a311b9f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:11.588767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76004a3d-bbd0-414f-ade6-c1bea6ddc174 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:11.683908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:11.688237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c8c88-cd2c-4898-a090-78fb7a311b9f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:11.690019 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76004a3d-bbd0-414f-ade6-c1bea6ddc174 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:11.722743 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984d1af-9d28-4936-b76e-30072e36a537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:11.723374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984d1af-9d28-4936-b76e-30072e36a537 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:11.723648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984d1af-9d28-4936-b76e-30072e36a537 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:11.723915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984d1af-9d28-4936-b76e-30072e36a537 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:11.724012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def4ba1-e9b0-4086-82f1-5a2dc9b7601d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:11.724513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def4ba1-e9b0-4086-82f1-5a2dc9b7601d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:11.724824 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def4ba1-e9b0-4086-82f1-5a2dc9b7601d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:11.725091 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def4ba1-e9b0-4086-82f1-5a2dc9b7601d None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:11.725850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def4ba1-e9b0-4086-82f1-5a2dc9b7601d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:11.726050 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def4ba1-e9b0-4086-82f1-5a2dc9b7601d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:11.727804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def4ba1-e9b0-4086-82f1-5a2dc9b7601d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:11.731013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984d1af-9d28-4936-b76e-30072e36a537 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:11.732479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984d1af-9d28-4936-b76e-30072e36a537 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:11.736153 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 541/1078] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:06:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:11.741877 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def4ba1-e9b0-4086-82f1-5a2dc9b7601d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-e984d1af-9d28-4936-b76e-30072e36a537 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:06:11.744205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:11.745180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:11.746058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:06:11.746058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:06:11.746058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:06:11.746058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:06:11.746058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:06:11.746058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:06:11.746058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:06:11.746058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:11.746058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 538/1079] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:06:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:11.750980 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984d1af-9d28-4936-b76e-30072e36a537 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:11.759845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:11.842535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def4ba1-e9b0-4086-82f1-5a2dc9b7601d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:11.851426 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984d1af-9d28-4936-b76e-30072e36a537 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:11.875872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fc3e7-9f98-4011-b31b-81b74479410e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:11.876497 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fc3e7-9f98-4011-b31b-81b74479410e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:11.876800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fc3e7-9f98-4011-b31b-81b74479410e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:11.877066 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fc3e7-9f98-4011-b31b-81b74479410e None admin] PATH_INFO: `/v3/users/e0b2280d4e72447d9efc4a3ee38628fe` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:11.878220 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fc3e7-9f98-4011-b31b-81b74479410e None admin] RBAC: Authorizing `identity:get_user(user_id=e0b2280d4e72447d9efc4a3ee38628fe)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:11.879642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fc3e7-9f98-4011-b31b-81b74479410e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:11.881250 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 542/1080] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:06:11 2022] GET /identity/v3/users/e0b2280d4e72447d9efc4a3ee38628fe => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:11.885901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018607f5-599f-4b62-b639-0c8eb0985017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:11.886501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018607f5-599f-4b62-b639-0c8eb0985017 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:11.886768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018607f5-599f-4b62-b639-0c8eb0985017 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:11.887030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018607f5-599f-4b62-b639-0c8eb0985017 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:11.887191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fc3e7-9f98-4011-b31b-81b74479410e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:11.887706 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018607f5-599f-4b62-b639-0c8eb0985017 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:11.887998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018607f5-599f-4b62-b639-0c8eb0985017 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:11.889449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018607f5-599f-4b62-b639-0c8eb0985017 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:11.901618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 539/1081] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:06:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:11.907583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018607f5-599f-4b62-b639-0c8eb0985017 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:11.915695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:11.975569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:11.990541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fc3e7-9f98-4011-b31b-81b74479410e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:12.010677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018607f5-599f-4b62-b639-0c8eb0985017 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:12.024786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0fe63-38c3-409e-ab14-6e82abb6d21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:12.025411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0fe63-38c3-409e-ab14-6e82abb6d21e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:12.025690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0fe63-38c3-409e-ab14-6e82abb6d21e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:12.025955 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0fe63-38c3-409e-ab14-6e82abb6d21e None admin] PATH_INFO: `/v3/projects/638cc303fbcb464d81a07c7cbf57c2da` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:12.027084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0fe63-38c3-409e-ab14-6e82abb6d21e None admin] RBAC: Authorizing `identity:get_project(project_id=638cc303fbcb464d81a07c7cbf57c2da)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:12.028816 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0fe63-38c3-409e-ab14-6e82abb6d21e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:12.030477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 543/1082] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:06:11 2022] GET /identity/v3/projects/638cc303fbcb464d81a07c7cbf57c2da => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:12.036851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0fe63-38c3-409e-ab14-6e82abb6d21e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:12.049303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d14d78-8d04-4783-9e21-7fdb8365bd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:12.049995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d14d78-8d04-4783-9e21-7fdb8365bd29 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:12.050253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d14d78-8d04-4783-9e21-7fdb8365bd29 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:12.050525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d14d78-8d04-4783-9e21-7fdb8365bd29 None admin] PATH_INFO: `/v3/projects/27ac51280046479e82faaf3c0b550f72/users/2af1ba390fff41a68ff1a0035901c318/roles/ea4eebb7bada4fe4a89fa7f71f4b09ce` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:12.052874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d14d78-8d04-4783-9e21-7fdb8365bd29 None admin] RBAC: Authorizing `identity:create_grant(project_id=27ac51280046479e82faaf3c0b550f72, user_id=2af1ba390fff41a68ff1a0035901c318, role_id=ea4eebb7bada4fe4a89fa7f71f4b09ce)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:12.054335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d14d78-8d04-4783-9e21-7fdb8365bd29 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:12.069719 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 540/1083] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:06:11 2022] PUT /identity/v3/projects/27ac51280046479e82faaf3c0b550f72/users/2af1ba390fff41a68ff1a0035901c318/roles/ea4eebb7bada4fe4a89fa7f71f4b09ce => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:06:12.143398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:12.151399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0fe63-38c3-409e-ab14-6e82abb6d21e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:12.188252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588b5d5-0afb-4716-bf56-c4c252bdc81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:12.188964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588b5d5-0afb-4716-bf56-c4c252bdc81b None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:12.189241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588b5d5-0afb-4716-bf56-c4c252bdc81b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:12.189503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588b5d5-0afb-4716-bf56-c4c252bdc81b None admin] PATH_INFO: `/v3/projects/638cc303fbcb464d81a07c7cbf57c2da/users/e0b2280d4e72447d9efc4a3ee38628fe/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:12.191516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2588b5d5-0afb-4716-bf56-c4c252bdc81b None admin] RBAC: Authorizing `identity:create_grant(project_id=638cc303fbcb464d81a07c7cbf57c2da, user_id=e0b2280d4e72447d9efc4a3ee38628fe, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:12.193064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2588b5d5-0afb-4716-bf56-c4c252bdc81b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:12.208224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 544/1084] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:06:12 2022] PUT /identity/v3/projects/638cc303fbcb464d81a07c7cbf57c2da/users/e0b2280d4e72447d9efc4a3ee38628fe/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:06:14.236572 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54993e1f-fbcf-404a-9b36-20b00dc5553f None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:14.237161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54993e1f-fbcf-404a-9b36-20b00dc5553f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:14.237161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54993e1f-fbcf-404a-9b36-20b00dc5553f None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:14.238488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 541/1085] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:14.243895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317a7e-122c-423b-9ce1-72e69013f312 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:14.244298 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317a7e-122c-423b-9ce1-72e69013f312 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:14.244412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5317a7e-122c-423b-9ce1-72e69013f312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:14.263635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f5317a7e-122c-423b-9ce1-72e69013f312 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:14.264850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f5317a7e-122c-423b-9ce1-72e69013f312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:14.337406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 545/1086] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:14.398134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2549ad-0a38-4a02-ab34-e00c317e6ed9 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:14.398397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2549ad-0a38-4a02-ab34-e00c317e6ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:14.398743 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2549ad-0a38-4a02-ab34-e00c317e6ed9 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:14.400178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 542/1087] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:14.405583 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0395e0-7fcb-48c0-a69e-bf1c06753cd2 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:14.405872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0395e0-7fcb-48c0-a69e-bf1c06753cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:14.406161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0395e0-7fcb-48c0-a69e-bf1c06753cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:14.425041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-fd0395e0-7fcb-48c0-a69e-bf1c06753cd2 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:14.426208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fd0395e0-7fcb-48c0-a69e-bf1c06753cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:14.497627 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 546/1088] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:14.598873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2549ad-0a38-4a02-ab34-e00c317e6ed9 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:14.704368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2549ad-0a38-4a02-ab34-e00c317e6ed9 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:14.729904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:14.740149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd571d0e-c501-429f-bbaf-d9a84f199987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:14.740823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd571d0e-c501-429f-bbaf-d9a84f199987 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:14.741092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd571d0e-c501-429f-bbaf-d9a84f199987 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:14.741348 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd571d0e-c501-429f-bbaf-d9a84f199987 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:14.751306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd571d0e-c501-429f-bbaf-d9a84f199987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-bd571d0e-c501-429f-bbaf-d9a84f199987 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:06:14.756004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:14.757009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:14.758432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:14.758432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:14.758432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:14.758432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:14.758432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:06:14.758432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:06:14.758432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:06:14.758432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:06:14.758432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:06:14.758432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:06:14.758432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 15:06:14.758432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:14.758432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 543/1089] 10.222.0.75 () {56 vars in 1120 bytes} [Fri Sep 23 15:06:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:14.763392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0395e0-7fcb-48c0-a69e-bf1c06753cd2 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:14.764348 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd571d0e-c501-429f-bbaf-d9a84f199987 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:14.781052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:14.866351 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd571d0e-c501-429f-bbaf-d9a84f199987 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:14.866685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0395e0-7fcb-48c0-a69e-bf1c06753cd2 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:14.899416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a542bb-7a11-4796-b9a4-1f1f32cbec97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:14.900025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a542bb-7a11-4796-b9a4-1f1f32cbec97 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:14.900293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a542bb-7a11-4796-b9a4-1f1f32cbec97 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:14.900604 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a542bb-7a11-4796-b9a4-1f1f32cbec97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:14.901985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2c97b-6782-4700-a5a0-c932d8faca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:14.902623 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2c97b-6782-4700-a5a0-c932d8faca2d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:14.903143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2c97b-6782-4700-a5a0-c932d8faca2d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:14.903143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2c97b-6782-4700-a5a0-c932d8faca2d None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:14.903798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2c97b-6782-4700-a5a0-c932d8faca2d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:14.904091 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2c97b-6782-4700-a5a0-c932d8faca2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:14.905474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2c97b-6782-4700-a5a0-c932d8faca2d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:14.910272 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a542bb-7a11-4796-b9a4-1f1f32cbec97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:06:14.913854 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 547/1090] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Sep 23 15:06:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-39a542bb-7a11-4796-b9a4-1f1f32cbec97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:06:14.915009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:14.916145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:14.917430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:14.917430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:14.917430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:14.917430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:14.917430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:06:14.917430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:06:14.917430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:06:14.917430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:06:14.917430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:06:14.917430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:06:14.917430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:06:14.917430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:14.917430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 544/1091] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:14.920420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2c97b-6782-4700-a5a0-c932d8faca2d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:14.924134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a542bb-7a11-4796-b9a4-1f1f32cbec97 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:14.973170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:15.002481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:15.026138 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2c97b-6782-4700-a5a0-c932d8faca2d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:15.026472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a542bb-7a11-4796-b9a4-1f1f32cbec97 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:15.063840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623080f9-736d-4347-9b2b-a48afbeb1459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:15.064463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623080f9-736d-4347-9b2b-a48afbeb1459 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:15.064677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83b550-7fac-4572-925e-47b1914611d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:15.064789 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623080f9-736d-4347-9b2b-a48afbeb1459 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:15.065008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623080f9-736d-4347-9b2b-a48afbeb1459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:15.065490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83b550-7fac-4572-925e-47b1914611d3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:15.065765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83b550-7fac-4572-925e-47b1914611d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:15.066027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83b550-7fac-4572-925e-47b1914611d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:15.066696 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83b550-7fac-4572-925e-47b1914611d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:15.066994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83b550-7fac-4572-925e-47b1914611d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:15.068364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83b550-7fac-4572-925e-47b1914611d3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:15.076763 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 545/1092] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:15.077075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623080f9-736d-4347-9b2b-a48afbeb1459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:15.078551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623080f9-736d-4347-9b2b-a48afbeb1459 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:15.082782 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83b550-7fac-4572-925e-47b1914611d3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-623080f9-736d-4347-9b2b-a48afbeb1459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:06:15.092862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:15.093927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:06:15.094978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:06:15.094978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:15.094978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:15.094978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 548/1093] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:15.099539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623080f9-736d-4347-9b2b-a48afbeb1459 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:15.184236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83b550-7fac-4572-925e-47b1914611d3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:15.192996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:15.209225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623080f9-736d-4347-9b2b-a48afbeb1459 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:15.218741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44746462-502c-4dac-8497-d2051d66e6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:15.219369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44746462-502c-4dac-8497-d2051d66e6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:15.219646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44746462-502c-4dac-8497-d2051d66e6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:15.219913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44746462-502c-4dac-8497-d2051d66e6d1 None admin] PATH_INFO: `/v3/users/e0b2280d4e72447d9efc4a3ee38628fe` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:15.221053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44746462-502c-4dac-8497-d2051d66e6d1 None admin] RBAC: Authorizing `identity:get_user(user_id=e0b2280d4e72447d9efc4a3ee38628fe)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:15.222489 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44746462-502c-4dac-8497-d2051d66e6d1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:15.224061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 546/1094] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:06:15 2022] GET /identity/v3/users/e0b2280d4e72447d9efc4a3ee38628fe => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:15.230498 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44746462-502c-4dac-8497-d2051d66e6d1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:15.236775 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:15.245502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a1c43-0a14-4a0b-b13b-151f81639451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:15.246097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a1c43-0a14-4a0b-b13b-151f81639451 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:15.246371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a1c43-0a14-4a0b-b13b-151f81639451 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:15.246636 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a1c43-0a14-4a0b-b13b-151f81639451 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:15.247399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a1c43-0a14-4a0b-b13b-151f81639451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:15.247697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a1c43-0a14-4a0b-b13b-151f81639451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:15.249098 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a1c43-0a14-4a0b-b13b-151f81639451 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:15.261816 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 549/1095] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:15.267475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a1c43-0a14-4a0b-b13b-151f81639451 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:15.332461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44746462-502c-4dac-8497-d2051d66e6d1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:15.366569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a1c43-0a14-4a0b-b13b-151f81639451 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:15.367584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a065d5-30c9-41c7-8d46-8a8f914c6357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:15.368282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a065d5-30c9-41c7-8d46-8a8f914c6357 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:15.368555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a065d5-30c9-41c7-8d46-8a8f914c6357 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:15.368842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a065d5-30c9-41c7-8d46-8a8f914c6357 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:15.401967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf42bc-c195-4d3a-83aa-079b54bb7938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:15.402588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf42bc-c195-4d3a-83aa-079b54bb7938 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:15.402859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf42bc-c195-4d3a-83aa-079b54bb7938 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:15.403122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf42bc-c195-4d3a-83aa-079b54bb7938 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:15.410258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf42bc-c195-4d3a-83aa-079b54bb7938 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:15.411669 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf42bc-c195-4d3a-83aa-079b54bb7938 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:15.412128 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-1bbf42bc-c195-4d3a-83aa-079b54bb7938 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:06:15.424010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:15.425068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:15.426024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:15.426024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:15.426024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:06:15.426024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:06:15.426024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:06:15.426024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:06:15.426024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:06:15.426024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:06:15.426024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 15:06:15.426024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:15.426024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 550/1096] 10.222.0.75 () {56 vars in 1093 bytes} [Fri Sep 23 15:06:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:06:15.431454 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf42bc-c195-4d3a-83aa-079b54bb7938 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:15.438369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a065d5-30c9-41c7-8d46-8a8f914c6357 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:15.439819 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a065d5-30c9-41c7-8d46-8a8f914c6357 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:15.479736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:15.532735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf42bc-c195-4d3a-83aa-079b54bb7938 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:15.544625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 547/1097] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:06:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 23 15:06:15.550303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a065d5-30c9-41c7-8d46-8a8f914c6357 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:15.569345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc95531-c9cb-4ba0-934e-fb46e3f67ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:15.569958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc95531-c9cb-4ba0-934e-fb46e3f67ffc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:15.570232 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc95531-c9cb-4ba0-934e-fb46e3f67ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:15.570571 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc95531-c9cb-4ba0-934e-fb46e3f67ffc None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:15.571402 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc95531-c9cb-4ba0-934e-fb46e3f67ffc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:15.571703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc95531-c9cb-4ba0-934e-fb46e3f67ffc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:15.573168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc95531-c9cb-4ba0-934e-fb46e3f67ffc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:15.590352 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 551/1098] 10.222.0.75 () {56 vars in 1133 bytes} [Fri Sep 23 15:06:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:15.596032 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc95531-c9cb-4ba0-934e-fb46e3f67ffc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:15.626146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:15.649953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a065d5-30c9-41c7-8d46-8a8f914c6357 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:15.686519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83480a-744b-47fe-afa9-4b803750ec6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:15.687144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83480a-744b-47fe-afa9-4b803750ec6b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:15.687412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83480a-744b-47fe-afa9-4b803750ec6b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:15.687675 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83480a-744b-47fe-afa9-4b803750ec6b None admin] PATH_INFO: `/v3/projects/638cc303fbcb464d81a07c7cbf57c2da` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:15.688993 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83480a-744b-47fe-afa9-4b803750ec6b None admin] RBAC: Authorizing `identity:get_project(project_id=638cc303fbcb464d81a07c7cbf57c2da)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:15.690427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83480a-744b-47fe-afa9-4b803750ec6b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:15.692091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 548/1099] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:06:15 2022] GET /identity/v3/projects/638cc303fbcb464d81a07c7cbf57c2da => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:15.698901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83480a-744b-47fe-afa9-4b803750ec6b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:15.702110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc95531-c9cb-4ba0-934e-fb46e3f67ffc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:15.711598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:15.742932 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126964c-0971-4135-832b-68e2b0e34211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:15.743820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126964c-0971-4135-832b-68e2b0e34211 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:15.744095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126964c-0971-4135-832b-68e2b0e34211 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:15.744363 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126964c-0971-4135-832b-68e2b0e34211 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:15.807648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83480a-744b-47fe-afa9-4b803750ec6b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:15.818928 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126964c-0971-4135-832b-68e2b0e34211 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:15.820396 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126964c-0971-4135-832b-68e2b0e34211 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:15.842399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202ceb8-902e-4648-ac70-b077711826b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:15.842994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202ceb8-902e-4648-ac70-b077711826b7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:15.843263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202ceb8-902e-4648-ac70-b077711826b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:15.843535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202ceb8-902e-4648-ac70-b077711826b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:15.844248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202ceb8-902e-4648-ac70-b077711826b7 None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=638cc303fbcb464d81a07c7cbf57c2da, user.id=e0b2280d4e72447d9efc4a3ee38628fe) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:15.844643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202ceb8-902e-4648-ac70-b077711826b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:15.846358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202ceb8-902e-4648-ac70-b077711826b7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:15.856624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 549/1100] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:06:15 2022] GET /identity/v3/role_assignments?user.id=e0b2280d4e72447d9efc4a3ee38628fe&scope.project.id=638cc303fbcb464d81a07c7cbf57c2da&role.id=ae444a489623434999d758b71ae2dc87 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:15.938434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 552/1101] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:06:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 23 15:06:15.945610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202ceb8-902e-4648-ac70-b077711826b7 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:15.959095 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:16.071451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202ceb8-902e-4648-ac70-b077711826b7 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:16.108060 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e22356-960d-4af3-a63a-5271a77087e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:16.108712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e22356-960d-4af3-a63a-5271a77087e5 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:16.109292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e22356-960d-4af3-a63a-5271a77087e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:16.109552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e22356-960d-4af3-a63a-5271a77087e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:16.120867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e22356-960d-4af3-a63a-5271a77087e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:16.122346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e22356-960d-4af3-a63a-5271a77087e5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-f9e22356-960d-4af3-a63a-5271a77087e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:06:16.137206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:16.138180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:06:16.139068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:06:16.139068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:06:16.139068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:16.140770 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 550/1102] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:16.144397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126964c-0971-4135-832b-68e2b0e34211 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:16.154705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:16.248559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126964c-0971-4135-832b-68e2b0e34211 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:16.283184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e93d83-4ea1-4a27-af15-32cda0a70cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:16.283775 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e93d83-4ea1-4a27-af15-32cda0a70cdb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:16.284043 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e93d83-4ea1-4a27-af15-32cda0a70cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:16.284307 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e93d83-4ea1-4a27-af15-32cda0a70cdb None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:16.284977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e93d83-4ea1-4a27-af15-32cda0a70cdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:16.285256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e93d83-4ea1-4a27-af15-32cda0a70cdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:16.286598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e93d83-4ea1-4a27-af15-32cda0a70cdb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:16.298109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 553/1103] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:16.303907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e22356-960d-4af3-a63a-5271a77087e5 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:16.366204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:16.412438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e22356-960d-4af3-a63a-5271a77087e5 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:16.449742 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c3247-c097-42dc-93f7-389eedb6a60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:16.450367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c3247-c097-42dc-93f7-389eedb6a60a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:16.450638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c3247-c097-42dc-93f7-389eedb6a60a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:16.450900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c3247-c097-42dc-93f7-389eedb6a60a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:16.457717 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c3247-c097-42dc-93f7-389eedb6a60a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:16.459152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c3247-c097-42dc-93f7-389eedb6a60a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-714c3247-c097-42dc-93f7-389eedb6a60a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:06:16.470773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:16.471778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:16.472687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:06:16.472687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:06:16.472687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:06:16.472687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:06:16.472687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:06:16.472687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:06:16.472687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:06:16.472687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:16.472687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 551/1104] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:06:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:16.478332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e93d83-4ea1-4a27-af15-32cda0a70cdb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:16.551160 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:16.587947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e93d83-4ea1-4a27-af15-32cda0a70cdb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:16.622973 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34afdc5-af04-4454-ad4e-983a276deffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:16.623488 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34afdc5-af04-4454-ad4e-983a276deffd None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:16.623753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34afdc5-af04-4454-ad4e-983a276deffd None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:16.624013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34afdc5-af04-4454-ad4e-983a276deffd None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:16.624714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34afdc5-af04-4454-ad4e-983a276deffd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:16.625012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34afdc5-af04-4454-ad4e-983a276deffd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:16.626401 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34afdc5-af04-4454-ad4e-983a276deffd None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:16.638347 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 554/1105] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:06:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:16.644478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c3247-c097-42dc-93f7-389eedb6a60a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:16.747821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c3247-c097-42dc-93f7-389eedb6a60a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:16.768395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:16.785870 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d225d-4017-4e74-8136-7dbe0bf2ce3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:16.786482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d225d-4017-4e74-8136-7dbe0bf2ce3e None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:16.786753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d225d-4017-4e74-8136-7dbe0bf2ce3e None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:16.787015 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d225d-4017-4e74-8136-7dbe0bf2ce3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:16.787743 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d225d-4017-4e74-8136-7dbe0bf2ce3e None admin] RBAC: Adding query filter params (role.id=ea4eebb7bada4fe4a89fa7f71f4b09ce, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=2af1ba390fff41a68ff1a0035901c318) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:16.788044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d225d-4017-4e74-8136-7dbe0bf2ce3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:16.790116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d225d-4017-4e74-8136-7dbe0bf2ce3e None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:16.796007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 552/1106] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:06:16 2022] GET /identity/v3/role_assignments?user.id=2af1ba390fff41a68ff1a0035901c318&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=ea4eebb7bada4fe4a89fa7f71f4b09ce => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:18.055706 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e7527-d0fc-4b04-9496-2957791f3278 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:18.056262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e7527-d0fc-4b04-9496-2957791f3278 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:18.056262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e7527-d0fc-4b04-9496-2957791f3278 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:18.058375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 555/1107] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:18.066586 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652017c6-2e2a-4818-9117-c826b624f800 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:18.067040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652017c6-2e2a-4818-9117-c826b624f800 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:18.067129 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652017c6-2e2a-4818-9117-c826b624f800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:18.087546 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-652017c6-2e2a-4818-9117-c826b624f800 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:18.088830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-652017c6-2e2a-4818-9117-c826b624f800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:18.159921 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 553/1108] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:06:18.431004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e7527-d0fc-4b04-9496-2957791f3278 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:18.550974 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e7527-d0fc-4b04-9496-2957791f3278 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:18.578886 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:18.589156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed0c82c-c39a-407d-ac9c-57d68f1f774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:18.589780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0c82c-c39a-407d-ac9c-57d68f1f774c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:18.590056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0c82c-c39a-407d-ac9c-57d68f1f774c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:18.590326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0c82c-c39a-407d-ac9c-57d68f1f774c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:18.591548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0c82c-c39a-407d-ac9c-57d68f1f774c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:18.593008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed0c82c-c39a-407d-ac9c-57d68f1f774c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:18.594716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 556/1109] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:18.603580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652017c6-2e2a-4818-9117-c826b624f800 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:18.706309 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652017c6-2e2a-4818-9117-c826b624f800 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:18.722995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:18.743890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd896867-09a2-42a9-ae58-9746ca848417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:18.744496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd896867-09a2-42a9-ae58-9746ca848417 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:18.744787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd896867-09a2-42a9-ae58-9746ca848417 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:18.745045 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd896867-09a2-42a9-ae58-9746ca848417 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:18.745692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd896867-09a2-42a9-ae58-9746ca848417 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:18.747108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd896867-09a2-42a9-ae58-9746ca848417 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:18.770866 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 554/1110] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:06:18 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:06:20.938568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b670227-6cb3-4de3-963e-de1adea10913 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:20.939402 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b670227-6cb3-4de3-963e-de1adea10913 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:20.939402 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b670227-6cb3-4de3-963e-de1adea10913 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:20.941664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 557/1111] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:20.950784 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f42cb0-105c-4970-ac47-363ffe9018b2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:20.951252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f42cb0-105c-4970-ac47-363ffe9018b2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:20.951252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f42cb0-105c-4970-ac47-363ffe9018b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:20.971311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-42f42cb0-105c-4970-ac47-363ffe9018b2 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:20.972474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-42f42cb0-105c-4970-ac47-363ffe9018b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:21.046234 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 555/1112] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:06:21.324216 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b670227-6cb3-4de3-963e-de1adea10913 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:21.442630 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b670227-6cb3-4de3-963e-de1adea10913 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:21.479478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f06c31-171b-4fb3-b0c0-08a76948e8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:21.480103 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06c31-171b-4fb3-b0c0-08a76948e8ff None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:21.480371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06c31-171b-4fb3-b0c0-08a76948e8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:21.480635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06c31-171b-4fb3-b0c0-08a76948e8ff None admin] PATH_INFO: `/v3/roles/704c67b455b54fb48089faef4aba8ccd` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:21.482160 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f06c31-171b-4fb3-b0c0-08a76948e8ff None admin] RBAC: Authorizing `identity:get_role(role_id=704c67b455b54fb48089faef4aba8ccd)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:21.483511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f06c31-171b-4fb3-b0c0-08a76948e8ff None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:21.485223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 558/1113] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:06:21 2022] GET /identity/v3/roles/704c67b455b54fb48089faef4aba8ccd => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:21.491620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f42cb0-105c-4970-ac47-363ffe9018b2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:21.603924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:21.607001 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f42cb0-105c-4970-ac47-363ffe9018b2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:21.648726 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd323f10-c289-4466-920f-b74fcde6d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:21.649355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd323f10-c289-4466-920f-b74fcde6d350 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:21.649621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd323f10-c289-4466-920f-b74fcde6d350 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:21.649882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd323f10-c289-4466-920f-b74fcde6d350 None admin] PATH_INFO: `/v3/users/5cd3d627b3c3478982a5f26b765c3d1f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:21.663152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd323f10-c289-4466-920f-b74fcde6d350 None admin] RBAC: Authorizing `identity:get_user(user_id=5cd3d627b3c3478982a5f26b765c3d1f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:21.664713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd323f10-c289-4466-920f-b74fcde6d350 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:21.666311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 556/1114] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:06:21 2022] GET /identity/v3/users/5cd3d627b3c3478982a5f26b765c3d1f => generated 286 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:21.673220 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f06c31-171b-4fb3-b0c0-08a76948e8ff None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:21.749127 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:21.795095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f06c31-171b-4fb3-b0c0-08a76948e8ff None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:21.831278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d1bb0-4b3a-4e6b-a6e2-b9a87373b067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:21.831871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d1bb0-4b3a-4e6b-a6e2-b9a87373b067 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:21.832140 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d1bb0-4b3a-4e6b-a6e2-b9a87373b067 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:21.832405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d1bb0-4b3a-4e6b-a6e2-b9a87373b067 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:21.905374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d1bb0-4b3a-4e6b-a6e2-b9a87373b067 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:21.906732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d1bb0-4b3a-4e6b-a6e2-b9a87373b067 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:21.967542 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:22.010834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 559/1115] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:06:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:06:22.017219 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd323f10-c289-4466-920f-b74fcde6d350 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:22.133583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd323f10-c289-4466-920f-b74fcde6d350 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:22.180247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dd8c4-b2de-416b-bc22-9bba8ea82f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:22.181003 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dd8c4-b2de-416b-bc22-9bba8ea82f10 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:22.181278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dd8c4-b2de-416b-bc22-9bba8ea82f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:22.181546 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dd8c4-b2de-416b-bc22-9bba8ea82f10 None admin] PATH_INFO: `/v3/projects/638cc303fbcb464d81a07c7cbf57c2da` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:22.182767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dd8c4-b2de-416b-bc22-9bba8ea82f10 None admin] RBAC: Authorizing `identity:get_project(project_id=638cc303fbcb464d81a07c7cbf57c2da)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:22.184202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dd8c4-b2de-416b-bc22-9bba8ea82f10 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:22.185849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 557/1116] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:06:22 2022] GET /identity/v3/projects/638cc303fbcb464d81a07c7cbf57c2da => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:22.196023 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d1bb0-4b3a-4e6b-a6e2-b9a87373b067 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:22.317747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d1bb0-4b3a-4e6b-a6e2-b9a87373b067 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:22.354989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84dd3f-708c-42d2-9008-e504fae3c854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:22.355619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84dd3f-708c-42d2-9008-e504fae3c854 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:22.355889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84dd3f-708c-42d2-9008-e504fae3c854 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:22.356154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84dd3f-708c-42d2-9008-e504fae3c854 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:22.356906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84dd3f-708c-42d2-9008-e504fae3c854 None admin] RBAC: Adding query filter params (role.id=704c67b455b54fb48089faef4aba8ccd, scope.project.id=638cc303fbcb464d81a07c7cbf57c2da, user.id=5cd3d627b3c3478982a5f26b765c3d1f) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:22.357213 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84dd3f-708c-42d2-9008-e504fae3c854 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:22.358639 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84dd3f-708c-42d2-9008-e504fae3c854 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:22.364960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 560/1117] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:06:22 2022] GET /identity/v3/role_assignments?user.id=5cd3d627b3c3478982a5f26b765c3d1f&scope.project.id=638cc303fbcb464d81a07c7cbf57c2da&role.id=704c67b455b54fb48089faef4aba8ccd => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:24.454516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5640d5-29f4-4673-a604-d35d3edd99c1 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:24.455347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5640d5-29f4-4673-a604-d35d3edd99c1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:24.455347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5640d5-29f4-4673-a604-d35d3edd99c1 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:24.457500 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 558/1118] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:24.466292 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4786-eb7a-4a05-af88-e969d78856ab None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:24.467005 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4786-eb7a-4a05-af88-e969d78856ab None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:24.467154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c4786-eb7a-4a05-af88-e969d78856ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:24.498567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-079c4786-eb7a-4a05-af88-e969d78856ab None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:24.500371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-079c4786-eb7a-4a05-af88-e969d78856ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:24.526967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:24.581995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 561/1119] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:24.848358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5640d5-29f4-4673-a604-d35d3edd99c1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:24.886366 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:24.965567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5640d5-29f4-4673-a604-d35d3edd99c1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:25.006987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894b4c5-42ce-4395-8ef7-f1b9adfeaf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:25.007612 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894b4c5-42ce-4395-8ef7-f1b9adfeaf28 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:25.007865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894b4c5-42ce-4395-8ef7-f1b9adfeaf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:25.008154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894b4c5-42ce-4395-8ef7-f1b9adfeaf28 None admin] PATH_INFO: `/v3/roles/704c67b455b54fb48089faef4aba8ccd` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:25.009656 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894b4c5-42ce-4395-8ef7-f1b9adfeaf28 None admin] RBAC: Authorizing `identity:get_role(role_id=704c67b455b54fb48089faef4aba8ccd)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:25.011026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894b4c5-42ce-4395-8ef7-f1b9adfeaf28 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:25.012402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 559/1120] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:06:24 2022] GET /identity/v3/roles/704c67b455b54fb48089faef4aba8ccd => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:25.021836 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c4786-eb7a-4a05-af88-e969d78856ab None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:25.130689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c4786-eb7a-4a05-af88-e969d78856ab None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:25.167505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233469-2024-4438-871e-a90c0f2768f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:25.168128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233469-2024-4438-871e-a90c0f2768f0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:25.168397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233469-2024-4438-871e-a90c0f2768f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:25.168683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2233469-2024-4438-871e-a90c0f2768f0 None admin] PATH_INFO: `/v3/users/5cd3d627b3c3478982a5f26b765c3d1f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:25.171068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2233469-2024-4438-871e-a90c0f2768f0 None admin] RBAC: Authorizing `identity:get_user(user_id=5cd3d627b3c3478982a5f26b765c3d1f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:25.172846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2233469-2024-4438-871e-a90c0f2768f0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:25.175290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 562/1121] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:06:25 2022] GET /identity/v3/users/5cd3d627b3c3478982a5f26b765c3d1f => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:25.182364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894b4c5-42ce-4395-8ef7-f1b9adfeaf28 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:25.283231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:25.292559 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894b4c5-42ce-4395-8ef7-f1b9adfeaf28 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:25.330823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ac807-c4ff-41d7-83f3-6aaa839a2f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:25.331450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ac807-c4ff-41d7-83f3-6aaa839a2f43 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:25.331721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ac807-c4ff-41d7-83f3-6aaa839a2f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:25.331994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ac807-c4ff-41d7-83f3-6aaa839a2f43 None admin] PATH_INFO: `/v3/projects/638cc303fbcb464d81a07c7cbf57c2da` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:25.333273 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ac807-c4ff-41d7-83f3-6aaa839a2f43 None admin] RBAC: Authorizing `identity:get_project(project_id=638cc303fbcb464d81a07c7cbf57c2da)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:25.334715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ac807-c4ff-41d7-83f3-6aaa839a2f43 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:25.336411 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 560/1122] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:06:25 2022] GET /identity/v3/projects/638cc303fbcb464d81a07c7cbf57c2da => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:25.343450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233469-2024-4438-871e-a90c0f2768f0 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:25.376035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:25.457306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2233469-2024-4438-871e-a90c0f2768f0 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:25.494617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bafe78-109f-471b-938c-bead10de00a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:25.495229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bafe78-109f-471b-938c-bead10de00a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:25.495505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bafe78-109f-471b-938c-bead10de00a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:25.495767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bafe78-109f-471b-938c-bead10de00a4 None admin] PATH_INFO: `/v3/projects/638cc303fbcb464d81a07c7cbf57c2da/users/5cd3d627b3c3478982a5f26b765c3d1f/roles/704c67b455b54fb48089faef4aba8ccd` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:25.498081 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bafe78-109f-471b-938c-bead10de00a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=638cc303fbcb464d81a07c7cbf57c2da, user_id=5cd3d627b3c3478982a5f26b765c3d1f, role_id=704c67b455b54fb48089faef4aba8ccd)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:25.499487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bafe78-109f-471b-938c-bead10de00a4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:25.520046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 563/1123] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:06:25 2022] PUT /identity/v3/projects/638cc303fbcb464d81a07c7cbf57c2da/users/5cd3d627b3c3478982a5f26b765c3d1f/roles/704c67b455b54fb48089faef4aba8ccd => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:06:27.656146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169804c-2247-4e13-8f3a-3b65ea7079cd None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:27.657070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169804c-2247-4e13-8f3a-3b65ea7079cd None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:27.657070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169804c-2247-4e13-8f3a-3b65ea7079cd None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:27.659185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 561/1124] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:27.665529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddc471-1730-4d12-93c9-4c58e28a2fdc None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:27.665950 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddc471-1730-4d12-93c9-4c58e28a2fdc None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:27.666062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddc471-1730-4d12-93c9-4c58e28a2fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:27.690493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-67ddc471-1730-4d12-93c9-4c58e28a2fdc None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:27.691695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-67ddc471-1730-4d12-93c9-4c58e28a2fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:27.760925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 564/1125] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:28.035032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169804c-2247-4e13-8f3a-3b65ea7079cd None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:28.162246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169804c-2247-4e13-8f3a-3b65ea7079cd None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:28.203017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2e02b-ddd8-44b1-a90e-68800174d115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:28.203633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2e02b-ddd8-44b1-a90e-68800174d115 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:28.203900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2e02b-ddd8-44b1-a90e-68800174d115 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:28.204159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2e02b-ddd8-44b1-a90e-68800174d115 None admin] PATH_INFO: `/v3/roles/704c67b455b54fb48089faef4aba8ccd` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:28.205594 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2e02b-ddd8-44b1-a90e-68800174d115 None admin] RBAC: Authorizing `identity:get_role(role_id=704c67b455b54fb48089faef4aba8ccd)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:28.206959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2e02b-ddd8-44b1-a90e-68800174d115 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:28.208318 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 562/1126] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:06:28 2022] GET /identity/v3/roles/704c67b455b54fb48089faef4aba8ccd => generated 223 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:28.217529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddc471-1730-4d12-93c9-4c58e28a2fdc None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:28.236942 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:28.327727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddc471-1730-4d12-93c9-4c58e28a2fdc None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:28.366433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b704f-ca6a-4bb4-8bf7-ebb155cdf06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:28.367052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b704f-ca6a-4bb4-8bf7-ebb155cdf06b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:28.367351 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b704f-ca6a-4bb4-8bf7-ebb155cdf06b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:28.367645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b704f-ca6a-4bb4-8bf7-ebb155cdf06b None admin] PATH_INFO: `/v3/users/5cd3d627b3c3478982a5f26b765c3d1f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:28.368818 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b704f-ca6a-4bb4-8bf7-ebb155cdf06b None admin] RBAC: Authorizing `identity:get_user(user_id=5cd3d627b3c3478982a5f26b765c3d1f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:28.370256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b704f-ca6a-4bb4-8bf7-ebb155cdf06b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:28.371880 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 565/1127] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:06:28 2022] GET /identity/v3/users/5cd3d627b3c3478982a5f26b765c3d1f => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:28.378686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2e02b-ddd8-44b1-a90e-68800174d115 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:28.413695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:28.493983 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2e02b-ddd8-44b1-a90e-68800174d115 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:28.530084 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1961596-8feb-4174-972d-d58798f4aa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:28.530702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1961596-8feb-4174-972d-d58798f4aa37 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:28.530982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1961596-8feb-4174-972d-d58798f4aa37 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:28.531241 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1961596-8feb-4174-972d-d58798f4aa37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:28.606587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1961596-8feb-4174-972d-d58798f4aa37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:28.607964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1961596-8feb-4174-972d-d58798f4aa37 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:28.642294 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:28.717237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 563/1128] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:06:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:06:28.723355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b704f-ca6a-4bb4-8bf7-ebb155cdf06b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:28.806939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:28.825825 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b704f-ca6a-4bb4-8bf7-ebb155cdf06b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:28.868886 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730b2b4-93ba-4ea5-9a7c-81186b29f80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:28.869688 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730b2b4-93ba-4ea5-9a7c-81186b29f80d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:28.870082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730b2b4-93ba-4ea5-9a7c-81186b29f80d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:28.870492 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730b2b4-93ba-4ea5-9a7c-81186b29f80d None admin] PATH_INFO: `/v3/projects/638cc303fbcb464d81a07c7cbf57c2da` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:28.872403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730b2b4-93ba-4ea5-9a7c-81186b29f80d None admin] RBAC: Authorizing `identity:get_project(project_id=638cc303fbcb464d81a07c7cbf57c2da)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:28.874673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730b2b4-93ba-4ea5-9a7c-81186b29f80d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:28.877192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 566/1129] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:06:28 2022] GET /identity/v3/projects/638cc303fbcb464d81a07c7cbf57c2da => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:28.886118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1961596-8feb-4174-972d-d58798f4aa37 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:28.997385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1961596-8feb-4174-972d-d58798f4aa37 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:29.033161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:29.042455 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525c057-ff4b-44d6-b8c0-5c98953969bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:29.043060 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525c057-ff4b-44d6-b8c0-5c98953969bc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:29.043333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525c057-ff4b-44d6-b8c0-5c98953969bc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:29.043601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525c057-ff4b-44d6-b8c0-5c98953969bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:29.044326 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525c057-ff4b-44d6-b8c0-5c98953969bc None admin] RBAC: Adding query filter params (role.id=704c67b455b54fb48089faef4aba8ccd, scope.project.id=638cc303fbcb464d81a07c7cbf57c2da, user.id=5cd3d627b3c3478982a5f26b765c3d1f) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:29.044669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525c057-ff4b-44d6-b8c0-5c98953969bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:29.046102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525c057-ff4b-44d6-b8c0-5c98953969bc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:29.051986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 564/1130] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:06:28 2022] GET /identity/v3/role_assignments?user.id=5cd3d627b3c3478982a5f26b765c3d1f&scope.project.id=638cc303fbcb464d81a07c7cbf57c2da&role.id=704c67b455b54fb48089faef4aba8ccd => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:31.224965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a844981-6a64-45c7-a9a6-deed4e3e4f69 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:31.225548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a844981-6a64-45c7-a9a6-deed4e3e4f69 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:31.225548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a844981-6a64-45c7-a9a6-deed4e3e4f69 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:31.227122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 567/1131] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:31.236196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26256716-dd37-49e9-bd35-ed2e022a66cf None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:31.236989 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26256716-dd37-49e9-bd35-ed2e022a66cf None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:31.237136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26256716-dd37-49e9-bd35-ed2e022a66cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:31.265192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-26256716-dd37-49e9-bd35-ed2e022a66cf None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:31.266375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-26256716-dd37-49e9-bd35-ed2e022a66cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:31.340217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 565/1132] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:31.604668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a844981-6a64-45c7-a9a6-deed4e3e4f69 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:31.719664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a844981-6a64-45c7-a9a6-deed4e3e4f69 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:31.755710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a251e-751c-4359-a416-c6941bcef881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:31.756328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a251e-751c-4359-a416-c6941bcef881 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:31.756618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a251e-751c-4359-a416-c6941bcef881 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:31.756894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a251e-751c-4359-a416-c6941bcef881 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:31.758266 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a251e-751c-4359-a416-c6941bcef881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:31.759680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a251e-751c-4359-a416-c6941bcef881 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:31.761323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 568/1133] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:31.768174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26256716-dd37-49e9-bd35-ed2e022a66cf None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:31.876804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26256716-dd37-49e9-bd35-ed2e022a66cf None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:31.908843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:31.924253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8f8a7-5f83-4fcd-b818-994e238e1d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:31.924907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8f8a7-5f83-4fcd-b818-994e238e1d68 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:31.925177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8f8a7-5f83-4fcd-b818-994e238e1d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:31.925440 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8f8a7-5f83-4fcd-b818-994e238e1d68 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:31.926088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8f8a7-5f83-4fcd-b818-994e238e1d68 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:31.927516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8f8a7-5f83-4fcd-b818-994e238e1d68 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:31.957472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 566/1134] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:06:31 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:06:34.220645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c647ea-5cb7-477c-965c-2d6679c049fc None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:34.221499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c647ea-5cb7-477c-965c-2d6679c049fc None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:34.221499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c647ea-5cb7-477c-965c-2d6679c049fc None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:34.223608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 569/1135] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:34.229989 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08632022-120b-422e-853e-2dbdf7d60187 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:34.230747 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08632022-120b-422e-853e-2dbdf7d60187 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:34.230891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08632022-120b-422e-853e-2dbdf7d60187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:34.252369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-08632022-120b-422e-853e-2dbdf7d60187 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:34.253596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-08632022-120b-422e-853e-2dbdf7d60187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:34.346232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 567/1136] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:34.624332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c647ea-5cb7-477c-965c-2d6679c049fc None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:34.654999 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:34.752372 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c647ea-5cb7-477c-965c-2d6679c049fc None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:34.786504 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f3a5e-3df1-4007-8c0d-025443cc3f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:34.787109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3a5e-3df1-4007-8c0d-025443cc3f25 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:34.787374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3a5e-3df1-4007-8c0d-025443cc3f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:34.787616 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3a5e-3df1-4007-8c0d-025443cc3f25 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:34.788860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f3a5e-3df1-4007-8c0d-025443cc3f25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:34.790312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f3a5e-3df1-4007-8c0d-025443cc3f25 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:34.791856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 570/1137] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:06:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:34.798376 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08632022-120b-422e-853e-2dbdf7d60187 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:34.918391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08632022-120b-422e-853e-2dbdf7d60187 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:34.926409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:34.956558 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d43c43-e328-4e38-b5ee-1f9689f60cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:34.957196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d43c43-e328-4e38-b5ee-1f9689f60cd5 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:34.957465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d43c43-e328-4e38-b5ee-1f9689f60cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:34.957732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d43c43-e328-4e38-b5ee-1f9689f60cd5 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:34.958387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d43c43-e328-4e38-b5ee-1f9689f60cd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:34.959809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d43c43-e328-4e38-b5ee-1f9689f60cd5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:34.988554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 568/1138] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:06:34 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:06:37.176401 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ddee7-c12b-4465-8093-48854dfd06b6 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:37.177014 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ddee7-c12b-4465-8093-48854dfd06b6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:37.177014 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ddee7-c12b-4465-8093-48854dfd06b6 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:37.178308 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 571/1139] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:37.185113 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3818-434f-4452-8893-e0d88695f8e7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:37.185802 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3818-434f-4452-8893-e0d88695f8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:37.185943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a3818-434f-4452-8893-e0d88695f8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:37.212810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b52a3818-434f-4452-8893-e0d88695f8e7 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:37.214027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b52a3818-434f-4452-8893-e0d88695f8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:37.291940 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 569/1140] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:37.568096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ddee7-c12b-4465-8093-48854dfd06b6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:37.659995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:37.677357 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ddee7-c12b-4465-8093-48854dfd06b6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:37.713210 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04aefad-54f4-4bbd-a348-d991b729319c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:37.713821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04aefad-54f4-4bbd-a348-d991b729319c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:37.714095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04aefad-54f4-4bbd-a348-d991b729319c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:37.714350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04aefad-54f4-4bbd-a348-d991b729319c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:37.725099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04aefad-54f4-4bbd-a348-d991b729319c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-c04aefad-54f4-4bbd-a348-d991b729319c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:06:37.729959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:37.731009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:37.732009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:37.732009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:37.732009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:37.732009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:37.732009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:06:37.732009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:06:37.732009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:06:37.732009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:06:37.732009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:06:37.732009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:06:37.732009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:06:37.732009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:37.732009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 572/1141] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:06:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:37.737738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a3818-434f-4452-8893-e0d88695f8e7 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:37.835202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:37.847890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a3818-434f-4452-8893-e0d88695f8e7 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:37.889537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26218a06-a6b9-477b-99a1-f2a98b82a9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:37.890136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26218a06-a6b9-477b-99a1-f2a98b82a9fd None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:37.890409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26218a06-a6b9-477b-99a1-f2a98b82a9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:37.890675 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26218a06-a6b9-477b-99a1-f2a98b82a9fd None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:37.891350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26218a06-a6b9-477b-99a1-f2a98b82a9fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:37.891652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26218a06-a6b9-477b-99a1-f2a98b82a9fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:37.893062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26218a06-a6b9-477b-99a1-f2a98b82a9fd None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:37.901259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 570/1142] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:06:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:37.910438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04aefad-54f4-4bbd-a348-d991b729319c None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:38.013498 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04aefad-54f4-4bbd-a348-d991b729319c None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:38.048211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e6541-2ee6-42a4-9f26-527c80bb97ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:38.048935 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e6541-2ee6-42a4-9f26-527c80bb97ba None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:38.049213 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e6541-2ee6-42a4-9f26-527c80bb97ba None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:38.049479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e6541-2ee6-42a4-9f26-527c80bb97ba None admin] PATH_INFO: `/v3/users/26f84acca25a4134a4d94c0f9d62e6a3` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:38.062859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e6541-2ee6-42a4-9f26-527c80bb97ba None admin] RBAC: Authorizing `identity:get_user(user_id=26f84acca25a4134a4d94c0f9d62e6a3)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:38.064319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e6541-2ee6-42a4-9f26-527c80bb97ba None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:38.065948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 573/1143] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:06:37 2022] GET /identity/v3/users/26f84acca25a4134a4d94c0f9d62e6a3 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:38.072639 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26218a06-a6b9-477b-99a1-f2a98b82a9fd None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:38.192341 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26218a06-a6b9-477b-99a1-f2a98b82a9fd None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:38.227713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc032ea3-d8fb-4de0-98fe-dbb3bd2d8ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:38.228310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc032ea3-d8fb-4de0-98fe-dbb3bd2d8ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:38.228581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc032ea3-d8fb-4de0-98fe-dbb3bd2d8ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:38.228879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc032ea3-d8fb-4de0-98fe-dbb3bd2d8ee1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:38.253229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:38.297727 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc032ea3-d8fb-4de0-98fe-dbb3bd2d8ee1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:38.299168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc032ea3-d8fb-4de0-98fe-dbb3bd2d8ee1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:38.407150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 571/1144] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:06:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:06:38.416544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e6541-2ee6-42a4-9f26-527c80bb97ba None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:38.421272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:38.524573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e6541-2ee6-42a4-9f26-527c80bb97ba None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:38.560137 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6c9c5-934b-4fe7-97ae-537440ab6849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:38.560783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6c9c5-934b-4fe7-97ae-537440ab6849 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:38.561079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6c9c5-934b-4fe7-97ae-537440ab6849 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:38.561294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6c9c5-934b-4fe7-97ae-537440ab6849 None admin] PATH_INFO: `/v3/projects/86a6b72daf8c462e86b5860059815467` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:38.563481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6c9c5-934b-4fe7-97ae-537440ab6849 None admin] RBAC: Authorizing `identity:get_project(project_id=86a6b72daf8c462e86b5860059815467)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:38.564899 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6c9c5-934b-4fe7-97ae-537440ab6849 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:38.566531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 574/1145] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:06:38 2022] GET /identity/v3/projects/86a6b72daf8c462e86b5860059815467 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:38.573162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc032ea3-d8fb-4de0-98fe-dbb3bd2d8ee1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:38.627774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:38.678380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc032ea3-d8fb-4de0-98fe-dbb3bd2d8ee1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:38.714259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac29eb-203e-4a76-932c-999242d542e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:38.714870 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac29eb-203e-4a76-932c-999242d542e1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:38.715148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac29eb-203e-4a76-932c-999242d542e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:38.715413 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac29eb-203e-4a76-932c-999242d542e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:38.716135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac29eb-203e-4a76-932c-999242d542e1 None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=86a6b72daf8c462e86b5860059815467, user.id=26f84acca25a4134a4d94c0f9d62e6a3) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:38.716440 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac29eb-203e-4a76-932c-999242d542e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:38.717936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac29eb-203e-4a76-932c-999242d542e1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:38.723552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 572/1146] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:06:38 2022] GET /identity/v3/role_assignments?user.id=26f84acca25a4134a4d94c0f9d62e6a3&scope.project.id=86a6b72daf8c462e86b5860059815467&role.id=ae444a489623434999d758b71ae2dc87 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:40.867414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a96c8d-95a3-4288-859c-2aea3b8e8ed3 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:40.867975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a96c8d-95a3-4288-859c-2aea3b8e8ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:40.868066 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a96c8d-95a3-4288-859c-2aea3b8e8ed3 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:40.869495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 575/1147] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:40.875913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acc930-36f6-4af8-939e-5dd2f395b2f5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:40.876345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acc930-36f6-4af8-939e-5dd2f395b2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:40.876453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acc930-36f6-4af8-939e-5dd2f395b2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:40.900038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b6acc930-36f6-4af8-939e-5dd2f395b2f5 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:40.901254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b6acc930-36f6-4af8-939e-5dd2f395b2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:40.973607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 573/1148] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:41.242807 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a96c8d-95a3-4288-859c-2aea3b8e8ed3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:41.334197 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:41.366749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a96c8d-95a3-4288-859c-2aea3b8e8ed3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:41.412601 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016bb3b-88ae-451b-862e-9ff35d90bba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:41.413245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016bb3b-88ae-451b-862e-9ff35d90bba3 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:41.413521 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016bb3b-88ae-451b-862e-9ff35d90bba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:41.413785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016bb3b-88ae-451b-862e-9ff35d90bba3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:41.424986 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016bb3b-88ae-451b-862e-9ff35d90bba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-3016bb3b-88ae-451b-862e-9ff35d90bba3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:06:41.429851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:41.431006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:41.432270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:41.432270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:41.432270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:41.432270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:41.432270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:06:41.432270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:06:41.432270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:06:41.432270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:06:41.432270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:06:41.432270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:06:41.432270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:06:41.432270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:41.432270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 576/1149] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:06:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:41.437987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acc930-36f6-4af8-939e-5dd2f395b2f5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:41.499646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:41.560964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acc930-36f6-4af8-939e-5dd2f395b2f5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:41.600947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31096d-6fcc-4b0d-b8ff-bf1dae47f74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:41.601556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31096d-6fcc-4b0d-b8ff-bf1dae47f74d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:41.601914 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31096d-6fcc-4b0d-b8ff-bf1dae47f74d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:41.602186 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31096d-6fcc-4b0d-b8ff-bf1dae47f74d None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:41.602857 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31096d-6fcc-4b0d-b8ff-bf1dae47f74d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:41.603164 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31096d-6fcc-4b0d-b8ff-bf1dae47f74d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:41.604540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31096d-6fcc-4b0d-b8ff-bf1dae47f74d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:41.613787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 574/1150] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:06:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:41.620178 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016bb3b-88ae-451b-862e-9ff35d90bba3 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:41.729689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016bb3b-88ae-451b-862e-9ff35d90bba3 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:41.759992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:41.766313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557f8e9-679d-41c0-a1da-3d15bdb7057b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:41.766929 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557f8e9-679d-41c0-a1da-3d15bdb7057b None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:41.767195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557f8e9-679d-41c0-a1da-3d15bdb7057b None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:41.767455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557f8e9-679d-41c0-a1da-3d15bdb7057b None admin] PATH_INFO: `/v3/users/26f84acca25a4134a4d94c0f9d62e6a3` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:41.768607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557f8e9-679d-41c0-a1da-3d15bdb7057b None admin] RBAC: Authorizing `identity:get_user(user_id=26f84acca25a4134a4d94c0f9d62e6a3)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:41.770057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9557f8e9-679d-41c0-a1da-3d15bdb7057b None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:41.771647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 577/1151] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:06:41 2022] GET /identity/v3/users/26f84acca25a4134a4d94c0f9d62e6a3 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:41.780952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31096d-6fcc-4b0d-b8ff-bf1dae47f74d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:41.888087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31096d-6fcc-4b0d-b8ff-bf1dae47f74d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:41.898250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:41.924671 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfaa14-56cd-44b3-ada3-379353e3b735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:41.925299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfaa14-56cd-44b3-ada3-379353e3b735 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:41.925564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfaa14-56cd-44b3-ada3-379353e3b735 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:41.925822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfaa14-56cd-44b3-ada3-379353e3b735 None admin] PATH_INFO: `/v3/projects/86a6b72daf8c462e86b5860059815467` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:41.926954 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfaa14-56cd-44b3-ada3-379353e3b735 None admin] RBAC: Authorizing `identity:get_project(project_id=86a6b72daf8c462e86b5860059815467)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:41.928393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfaa14-56cd-44b3-ada3-379353e3b735 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:41.930177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 575/1152] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:06:41 2022] GET /identity/v3/projects/86a6b72daf8c462e86b5860059815467 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:06:41.936738 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557f8e9-679d-41c0-a1da-3d15bdb7057b None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:42.036650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557f8e9-679d-41c0-a1da-3d15bdb7057b None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:42.071518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1392f-95c6-454d-9d91-922a597ce8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:42.072155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1392f-95c6-454d-9d91-922a597ce8d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:42.072424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1392f-95c6-454d-9d91-922a597ce8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:42.072719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1392f-95c6-454d-9d91-922a597ce8d2 None admin] PATH_INFO: `/v3/projects/86a6b72daf8c462e86b5860059815467/users/26f84acca25a4134a4d94c0f9d62e6a3/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:42.074877 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1392f-95c6-454d-9d91-922a597ce8d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=86a6b72daf8c462e86b5860059815467, user_id=26f84acca25a4134a4d94c0f9d62e6a3, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:42.076292 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc1392f-95c6-454d-9d91-922a597ce8d2 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:42.092249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 578/1153] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:06:41 2022] PUT /identity/v3/projects/86a6b72daf8c462e86b5860059815467/users/26f84acca25a4134a4d94c0f9d62e6a3/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:06:44.189568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf21dde-e845-42fd-bf98-d51f9a35603c None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:44.190454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf21dde-e845-42fd-bf98-d51f9a35603c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:44.190454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf21dde-e845-42fd-bf98-d51f9a35603c None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:44.192741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 576/1154] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:44.199006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112e49f-e277-46b8-8ff0-e80894bca1ee None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:44.199718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112e49f-e277-46b8-8ff0-e80894bca1ee None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:44.199867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112e49f-e277-46b8-8ff0-e80894bca1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:44.222920 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6112e49f-e277-46b8-8ff0-e80894bca1ee None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:44.224103 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6112e49f-e277-46b8-8ff0-e80894bca1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:44.281719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:44.294189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 579/1155] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:44.562277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf21dde-e845-42fd-bf98-d51f9a35603c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:44.676184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf21dde-e845-42fd-bf98-d51f9a35603c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:44.710821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b774eea-9926-49b3-8bd3-038cee3bb99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:44.711459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b774eea-9926-49b3-8bd3-038cee3bb99b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:44.711775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b774eea-9926-49b3-8bd3-038cee3bb99b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:44.712038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b774eea-9926-49b3-8bd3-038cee3bb99b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:44.723132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b774eea-9926-49b3-8bd3-038cee3bb99b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-0b774eea-9926-49b3-8bd3-038cee3bb99b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:06:44.728097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:44.729114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:44.730057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:44.730057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:44.730057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:44.730057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:44.730057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:06:44.730057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:06:44.730057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:06:44.730057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:06:44.730057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:06:44.730057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:06:44.730057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:06:44.730057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:06:44.730057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 577/1156] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:06:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:06:44.738260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112e49f-e277-46b8-8ff0-e80894bca1ee None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:44.845477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112e49f-e277-46b8-8ff0-e80894bca1ee None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:44.880512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007c060-dfe7-40aa-98c5-b53ca7b1c2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:44.881137 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007c060-dfe7-40aa-98c5-b53ca7b1c2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:44.881405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007c060-dfe7-40aa-98c5-b53ca7b1c2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:44.881666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007c060-dfe7-40aa-98c5-b53ca7b1c2c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:44.882332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007c060-dfe7-40aa-98c5-b53ca7b1c2c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:44.882628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007c060-dfe7-40aa-98c5-b53ca7b1c2c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:44.884107 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a007c060-dfe7-40aa-98c5-b53ca7b1c2c1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:44.892816 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 580/1157] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:06:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:44.901847 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b774eea-9926-49b3-8bd3-038cee3bb99b None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:44.955895 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:45.028885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b774eea-9926-49b3-8bd3-038cee3bb99b None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:45.063629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41af1f-43c9-4da3-a552-81b576bc2bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:45.064245 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41af1f-43c9-4da3-a552-81b576bc2bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:45.064516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41af1f-43c9-4da3-a552-81b576bc2bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:45.064819 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41af1f-43c9-4da3-a552-81b576bc2bb1 None admin] PATH_INFO: `/v3/users/26f84acca25a4134a4d94c0f9d62e6a3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:45.066073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41af1f-43c9-4da3-a552-81b576bc2bb1 None admin] RBAC: Authorizing `identity:get_user(user_id=26f84acca25a4134a4d94c0f9d62e6a3)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:45.067503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41af1f-43c9-4da3-a552-81b576bc2bb1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:45.069107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 578/1158] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:06:44 2022] GET /identity/v3/users/26f84acca25a4134a4d94c0f9d62e6a3 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:45.077805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007c060-dfe7-40aa-98c5-b53ca7b1c2c1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:45.132304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:45.182054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007c060-dfe7-40aa-98c5-b53ca7b1c2c1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:45.216613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77102afd-208b-4923-8999-693dbb3b98d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:45.217219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77102afd-208b-4923-8999-693dbb3b98d9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:45.217461 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77102afd-208b-4923-8999-693dbb3b98d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:45.217728 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77102afd-208b-4923-8999-693dbb3b98d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:45.286526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77102afd-208b-4923-8999-693dbb3b98d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:45.287940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77102afd-208b-4923-8999-693dbb3b98d9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:45.339547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:45.391824 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 581/1159] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:06:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 23 15:06:45.397869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41af1f-43c9-4da3-a552-81b576bc2bb1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:45.506889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41af1f-43c9-4da3-a552-81b576bc2bb1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:45.513753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:45.543925 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e557b28-f4c2-4a84-aedb-4d1f87f3b522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:45.544560 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e557b28-f4c2-4a84-aedb-4d1f87f3b522 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:45.544844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e557b28-f4c2-4a84-aedb-4d1f87f3b522 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:45.545136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e557b28-f4c2-4a84-aedb-4d1f87f3b522 None admin] PATH_INFO: `/v3/projects/86a6b72daf8c462e86b5860059815467` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:45.546568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e557b28-f4c2-4a84-aedb-4d1f87f3b522 None admin] RBAC: Authorizing `identity:get_project(project_id=86a6b72daf8c462e86b5860059815467)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:45.547748 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e557b28-f4c2-4a84-aedb-4d1f87f3b522 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:45.549416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 579/1160] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:06:45 2022] GET /identity/v3/projects/86a6b72daf8c462e86b5860059815467 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:45.555722 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77102afd-208b-4923-8999-693dbb3b98d9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:45.667835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77102afd-208b-4923-8999-693dbb3b98d9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:45.703690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0290b-0ee9-4018-a3a5-ddf2fe77f264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:45.704278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0290b-0ee9-4018-a3a5-ddf2fe77f264 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:45.704548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0290b-0ee9-4018-a3a5-ddf2fe77f264 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:45.704845 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0290b-0ee9-4018-a3a5-ddf2fe77f264 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:45.705561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0290b-0ee9-4018-a3a5-ddf2fe77f264 None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=86a6b72daf8c462e86b5860059815467, user.id=26f84acca25a4134a4d94c0f9d62e6a3) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:45.705864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0290b-0ee9-4018-a3a5-ddf2fe77f264 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:45.707284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0290b-0ee9-4018-a3a5-ddf2fe77f264 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:45.713079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 582/1161] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:06:45 2022] GET /identity/v3/role_assignments?user.id=26f84acca25a4134a4d94c0f9d62e6a3&scope.project.id=86a6b72daf8c462e86b5860059815467&role.id=ae444a489623434999d758b71ae2dc87 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:47.873655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcca9a8-46bc-43e7-9c2e-3bd1657811bc None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:47.874574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcca9a8-46bc-43e7-9c2e-3bd1657811bc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:47.874574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcca9a8-46bc-43e7-9c2e-3bd1657811bc None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:47.876652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 580/1162] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:47.883119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df1e43-2cc3-4a5e-8c14-ddad96e83f09 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:47.883555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df1e43-2cc3-4a5e-8c14-ddad96e83f09 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:47.883655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df1e43-2cc3-4a5e-8c14-ddad96e83f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:47.903484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-81df1e43-2cc3-4a5e-8c14-ddad96e83f09 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:47.904690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-81df1e43-2cc3-4a5e-8c14-ddad96e83f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:47.915762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:47.975095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 583/1163] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:48.250134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcca9a8-46bc-43e7-9c2e-3bd1657811bc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:48.366102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcca9a8-46bc-43e7-9c2e-3bd1657811bc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:48.403414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129a51e-6b4f-4669-8d0f-c6c2c4f88c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:48.404011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129a51e-6b4f-4669-8d0f-c6c2c4f88c5a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:48.404283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129a51e-6b4f-4669-8d0f-c6c2c4f88c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:48.404549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129a51e-6b4f-4669-8d0f-c6c2c4f88c5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:48.451725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:48.493220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129a51e-6b4f-4669-8d0f-c6c2c4f88c5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:48.494679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129a51e-6b4f-4669-8d0f-c6c2c4f88c5a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:48.604190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 581/1164] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:06:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:06:48.611015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df1e43-2cc3-4a5e-8c14-ddad96e83f09 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:48.731544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df1e43-2cc3-4a5e-8c14-ddad96e83f09 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:48.766482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8dd79c-9f46-4b21-97ee-511ff098a0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:48.767097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8dd79c-9f46-4b21-97ee-511ff098a0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:48.767369 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8dd79c-9f46-4b21-97ee-511ff098a0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:48.767631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8dd79c-9f46-4b21-97ee-511ff098a0d8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:48.774462 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8dd79c-9f46-4b21-97ee-511ff098a0d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:48.775902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8dd79c-9f46-4b21-97ee-511ff098a0d8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-9b8dd79c-9f46-4b21-97ee-511ff098a0d8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:06:48.787765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:06:48.788966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:06:48.789839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 584/1165] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Sep 23 15:06:48 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:06:48.797766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129a51e-6b4f-4669-8d0f-c6c2c4f88c5a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:48.879558 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:48.908756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129a51e-6b4f-4669-8d0f-c6c2c4f88c5a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:48.943522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a73fa-a6cf-4b7b-bfaf-2634aa2bf964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:48.944108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a73fa-a6cf-4b7b-bfaf-2634aa2bf964 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:48.944377 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a73fa-a6cf-4b7b-bfaf-2634aa2bf964 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:48.944648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a73fa-a6cf-4b7b-bfaf-2634aa2bf964 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:48.945299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a73fa-a6cf-4b7b-bfaf-2634aa2bf964 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:06:48.945591 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a73fa-a6cf-4b7b-bfaf-2634aa2bf964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:48.946943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a73fa-a6cf-4b7b-bfaf-2634aa2bf964 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:48.953990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 582/1166] 10.222.0.75 () {56 vars in 1115 bytes} [Fri Sep 23 15:06:48 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:51.139365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b0b78-bbed-44ae-9d29-9423f0f2a908 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:51.140264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b0b78-bbed-44ae-9d29-9423f0f2a908 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:51.140264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b0b78-bbed-44ae-9d29-9423f0f2a908 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:51.142324 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 585/1167] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:51.151544 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8c6cc-19e2-4d31-89e0-1782c442960a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:51.152276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8c6cc-19e2-4d31-89e0-1782c442960a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:51.152370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8c6cc-19e2-4d31-89e0-1782c442960a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:51.182471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f0d8c6cc-19e2-4d31-89e0-1782c442960a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:51.183648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f0d8c6cc-19e2-4d31-89e0-1782c442960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:51.257269 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 583/1168] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:51.519028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b0b78-bbed-44ae-9d29-9423f0f2a908 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:51.529304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:51.627534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b0b78-bbed-44ae-9d29-9423f0f2a908 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:51.661550 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307548ab-7ed7-4205-a20c-a6d7386d36f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:51.662131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307548ab-7ed7-4205-a20c-a6d7386d36f7 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:51.662397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307548ab-7ed7-4205-a20c-a6d7386d36f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:51.662659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307548ab-7ed7-4205-a20c-a6d7386d36f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:51.663329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307548ab-7ed7-4205-a20c-a6d7386d36f7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:51.664697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307548ab-7ed7-4205-a20c-a6d7386d36f7 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:51.686414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 586/1169] 10.222.0.75 () {60 vars in 1138 bytes} [Fri Sep 23 15:06:51 2022] POST /identity/v3/domains => generated 267 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:06:53.833793 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec42ff5-1318-4f9e-8513-f87bd3552485 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:53.834683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec42ff5-1318-4f9e-8513-f87bd3552485 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:53.834683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec42ff5-1318-4f9e-8513-f87bd3552485 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:53.836891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 584/1170] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:53.844891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665f865-64ea-4b40-90de-f956e77bf119 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:53.845350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665f865-64ea-4b40-90de-f956e77bf119 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:53.845456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e665f865-64ea-4b40-90de-f956e77bf119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:53.864545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e665f865-64ea-4b40-90de-f956e77bf119 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:53.865727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e665f865-64ea-4b40-90de-f956e77bf119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:53.928563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:53.937205 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 587/1171] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:06:54.199839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec42ff5-1318-4f9e-8513-f87bd3552485 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:54.280958 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:54.322271 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec42ff5-1318-4f9e-8513-f87bd3552485 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:54.356453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd6122-07a5-45e1-85d1-23d9ac75e143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:54.357104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd6122-07a5-45e1-85d1-23d9ac75e143 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:54.357371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd6122-07a5-45e1-85d1-23d9ac75e143 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:54.357628 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd6122-07a5-45e1-85d1-23d9ac75e143 None admin] PATH_INFO: `/v3/domains/0fc9aacb3b52422688ddcc921ba7386c` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:54.368492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd6122-07a5-45e1-85d1-23d9ac75e143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0fc9aacb3b52422688ddcc921ba7386c)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:54.370118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd6122-07a5-45e1-85d1-23d9ac75e143 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:54.371722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 585/1172] 10.222.0.75 () {56 vars in 1183 bytes} [Fri Sep 23 15:06:54 2022] GET /identity/v3/domains/0fc9aacb3b52422688ddcc921ba7386c => generated 267 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:54.378538 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665f865-64ea-4b40-90de-f956e77bf119 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:54.493528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e665f865-64ea-4b40-90de-f956e77bf119 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:54.544109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176f3a4-0463-4ffc-9735-59a3e52ddd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:54.544762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176f3a4-0463-4ffc-9735-59a3e52ddd2e None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:54.545036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176f3a4-0463-4ffc-9735-59a3e52ddd2e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:54.545680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176f3a4-0463-4ffc-9735-59a3e52ddd2e None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:54.545948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176f3a4-0463-4ffc-9735-59a3e52ddd2e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:54.547373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176f3a4-0463-4ffc-9735-59a3e52ddd2e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:54.577891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 588/1173] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:06:54 2022] POST /identity/v3/projects => generated 361 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:06:56.757627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3fdc9-947c-46b7-9470-3e236784b143 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:56.758511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3fdc9-947c-46b7-9470-3e236784b143 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:56.758511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3fdc9-947c-46b7-9470-3e236784b143 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:56.760789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 586/1174] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:56.766975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984cb86-fa02-4c76-a9e7-58bbc7c5d929 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:56.767425 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984cb86-fa02-4c76-a9e7-58bbc7c5d929 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:56.767518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984cb86-fa02-4c76-a9e7-58bbc7c5d929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:56.795319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5984cb86-fa02-4c76-a9e7-58bbc7c5d929 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:56.796533 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5984cb86-fa02-4c76-a9e7-58bbc7c5d929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:56.843248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:56.870323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 589/1175] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:06:57.132388 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3fdc9-947c-46b7-9470-3e236784b143 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:57.259931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3fdc9-947c-46b7-9470-3e236784b143 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:57.286122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:57.296026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fd2dd-7ffc-4093-88e2-39884caab21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:57.296683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fd2dd-7ffc-4093-88e2-39884caab21f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:57.296954 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fd2dd-7ffc-4093-88e2-39884caab21f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:57.297215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fd2dd-7ffc-4093-88e2-39884caab21f None admin] PATH_INFO: `/v3/domains/0fc9aacb3b52422688ddcc921ba7386c` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:57.298426 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fd2dd-7ffc-4093-88e2-39884caab21f None admin] RBAC: Authorizing `identity:get_domain(domain_id=0fc9aacb3b52422688ddcc921ba7386c)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:57.299884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fd2dd-7ffc-4093-88e2-39884caab21f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:57.301451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 587/1176] 10.222.0.75 () {56 vars in 1183 bytes} [Fri Sep 23 15:06:57 2022] GET /identity/v3/domains/0fc9aacb3b52422688ddcc921ba7386c => generated 267 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:06:57.307714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984cb86-fa02-4c76-a9e7-58bbc7c5d929 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:06:57.421778 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984cb86-fa02-4c76-a9e7-58bbc7c5d929 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:06:57.460181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a68af-04d6-4cee-9220-f64d372ac7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:06:57.461132 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a68af-04d6-4cee-9220-f64d372ac7be None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:57.461499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a68af-04d6-4cee-9220-f64d372ac7be None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:57.461890 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a68af-04d6-4cee-9220-f64d372ac7be None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:57.462993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a68af-04d6-4cee-9220-f64d372ac7be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:06:57.465334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a68af-04d6-4cee-9220-f64d372ac7be None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:06:57.496198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 590/1177] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:06:57 2022] POST /identity/v3/users => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:06:59.687184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e71fd5-15cd-4e4b-b095-bdafe8736a20 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:59.687747 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e71fd5-15cd-4e4b-b095-bdafe8736a20 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:59.687747 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e71fd5-15cd-4e4b-b095-bdafe8736a20 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:59.689084 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 588/1178] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:06:59.695458 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5aaa34-613d-4661-900f-ddd0d550db64 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:06:59.696188 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5aaa34-613d-4661-900f-ddd0d550db64 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:06:59.696353 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5aaa34-613d-4661-900f-ddd0d550db64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:06:59.716978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-3d5aaa34-613d-4661-900f-ddd0d550db64 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:06:59.718185 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3d5aaa34-613d-4661-900f-ddd0d550db64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:06:59.748581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:06:59.796984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 591/1179] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:06:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:07:00.062587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e71fd5-15cd-4e4b-b095-bdafe8736a20 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:00.173185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e71fd5-15cd-4e4b-b095-bdafe8736a20 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:00.206980 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d33f1-9112-45e3-98e6-a7939eb9c2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:00.207583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d33f1-9112-45e3-98e6-a7939eb9c2eb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:00.207848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d33f1-9112-45e3-98e6-a7939eb9c2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:00.208108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d33f1-9112-45e3-98e6-a7939eb9c2eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:00.218936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d33f1-9112-45e3-98e6-a7939eb9c2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-8f0d33f1-9112-45e3-98e6-a7939eb9c2eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:07:00.223623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:07:00.224753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:07:00.225759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:07:00.225759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:07:00.225759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:07:00.225759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:07:00.225759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:07:00.225759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:07:00.225759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:07:00.225759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:07:00.225759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:07:00.225759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:07:00.225759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:07:00.225759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:07:00.225759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 589/1180] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:07:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:07:00.231173 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5aaa34-613d-4661-900f-ddd0d550db64 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:00.331185 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5aaa34-613d-4661-900f-ddd0d550db64 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:00.366934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e0759-8428-4932-9889-d32f2c216a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:00.367531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e0759-8428-4932-9889-d32f2c216a0d None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:00.367798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e0759-8428-4932-9889-d32f2c216a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:00.368065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e0759-8428-4932-9889-d32f2c216a0d None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:00.369366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e0759-8428-4932-9889-d32f2c216a0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:07:00.369668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e0759-8428-4932-9889-d32f2c216a0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:00.371025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e0759-8428-4932-9889-d32f2c216a0d None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:00.378952 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 592/1181] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:07:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:07:00.385605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d33f1-9112-45e3-98e6-a7939eb9c2eb None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:00.456466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:00.505208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d33f1-9112-45e3-98e6-a7939eb9c2eb None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:00.540423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efeb10-039d-42e0-a6e5-d860ccd1edfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:00.541187 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efeb10-039d-42e0-a6e5-d860ccd1edfc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:00.541462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efeb10-039d-42e0-a6e5-d860ccd1edfc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:00.541735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3efeb10-039d-42e0-a6e5-d860ccd1edfc None admin] PATH_INFO: `/v3/users/df283df53ce749c9a0cecfa677d27d21` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:00.557947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3efeb10-039d-42e0-a6e5-d860ccd1edfc None admin] RBAC: Authorizing `identity:get_user(user_id=df283df53ce749c9a0cecfa677d27d21)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:00.559394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3efeb10-039d-42e0-a6e5-d860ccd1edfc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:00.561046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 590/1182] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:07:00 2022] GET /identity/v3/users/df283df53ce749c9a0cecfa677d27d21 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:07:00.570558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e0759-8428-4932-9889-d32f2c216a0d None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:00.612254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:00.695615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e0759-8428-4932-9889-d32f2c216a0d None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:00.730348 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f96f2-213e-40a2-baf1-561372cd7e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:00.730929 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f96f2-213e-40a2-baf1-561372cd7e16 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:00.731201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f96f2-213e-40a2-baf1-561372cd7e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:00.731455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f96f2-213e-40a2-baf1-561372cd7e16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:00.805518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f96f2-213e-40a2-baf1-561372cd7e16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:00.806978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f96f2-213e-40a2-baf1-561372cd7e16 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:00.838522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:00.911836 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 593/1183] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:07:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:07:00.918201 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efeb10-039d-42e0-a6e5-d860ccd1edfc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:01.024962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3efeb10-039d-42e0-a6e5-d860ccd1edfc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:01.038927 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:01.062060 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894c715-d186-471b-9e3b-1df6940a4528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:01.062701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894c715-d186-471b-9e3b-1df6940a4528 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:01.062975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894c715-d186-471b-9e3b-1df6940a4528 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:01.063244 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894c715-d186-471b-9e3b-1df6940a4528 None admin] PATH_INFO: `/v3/projects/b3b5ae30bafe47d1ac27e36b9f29f3ad` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:01.064422 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894c715-d186-471b-9e3b-1df6940a4528 None admin] RBAC: Authorizing `identity:get_project(project_id=b3b5ae30bafe47d1ac27e36b9f29f3ad)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:01.065892 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f894c715-d186-471b-9e3b-1df6940a4528 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:01.067517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 591/1184] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:07:00 2022] GET /identity/v3/projects/b3b5ae30bafe47d1ac27e36b9f29f3ad => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:07:01.074278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f96f2-213e-40a2-baf1-561372cd7e16 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:01.174078 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f96f2-213e-40a2-baf1-561372cd7e16 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:01.201415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:01.212443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee57872-cde4-4640-aca5-b1b09c97d158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:01.213073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee57872-cde4-4640-aca5-b1b09c97d158 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:01.213345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee57872-cde4-4640-aca5-b1b09c97d158 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:01.213594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee57872-cde4-4640-aca5-b1b09c97d158 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:01.214332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee57872-cde4-4640-aca5-b1b09c97d158 None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=b3b5ae30bafe47d1ac27e36b9f29f3ad, user.id=df283df53ce749c9a0cecfa677d27d21) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:07:01.214636 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee57872-cde4-4640-aca5-b1b09c97d158 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:01.216056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee57872-cde4-4640-aca5-b1b09c97d158 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:01.222193 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 594/1185] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:07:01 2022] GET /identity/v3/role_assignments?user.id=df283df53ce749c9a0cecfa677d27d21&scope.project.id=b3b5ae30bafe47d1ac27e36b9f29f3ad&role.id=ae444a489623434999d758b71ae2dc87 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:07:03.382396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495ed75-aff2-45ca-88e7-7f870e213ea9 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:03.382993 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495ed75-aff2-45ca-88e7-7f870e213ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:03.382993 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495ed75-aff2-45ca-88e7-7f870e213ea9 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:03.384264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 592/1186] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:07:03.392423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a01343-abdc-4e24-bc52-57603656b2f7 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:03.393206 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a01343-abdc-4e24-bc52-57603656b2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:03.393330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a01343-abdc-4e24-bc52-57603656b2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:03.421397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f3a01343-abdc-4e24-bc52-57603656b2f7 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:07:03.422605 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f3a01343-abdc-4e24-bc52-57603656b2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:07:03.497302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 595/1187] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:07:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:07:03.766232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495ed75-aff2-45ca-88e7-7f870e213ea9 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:03.882889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495ed75-aff2-45ca-88e7-7f870e213ea9 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:03.918814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87203446-3812-4554-b5fa-c2199b68e34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:03.919429 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87203446-3812-4554-b5fa-c2199b68e34a None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:03.919724 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87203446-3812-4554-b5fa-c2199b68e34a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:03.919986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87203446-3812-4554-b5fa-c2199b68e34a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:03.931087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87203446-3812-4554-b5fa-c2199b68e34a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-87203446-3812-4554-b5fa-c2199b68e34a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:07:03.936052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:07:03.937109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:07:03.938056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:07:03.938056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:07:03.938056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:07:03.938056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:07:03.938056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:07:03.938056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:07:03.938056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:07:03.938056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:07:03.938056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:07:03.938056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:07:03.938056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:07:03.938056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:07:03.938056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 593/1188] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:07:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:07:03.945806 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a01343-abdc-4e24-bc52-57603656b2f7 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:04.045436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a01343-abdc-4e24-bc52-57603656b2f7 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:04.065883 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:04.082724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935e194-e4e8-44f6-a21b-9b83f31ac013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:04.083318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935e194-e4e8-44f6-a21b-9b83f31ac013 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:04.083587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935e194-e4e8-44f6-a21b-9b83f31ac013 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:04.083852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935e194-e4e8-44f6-a21b-9b83f31ac013 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:04.084514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935e194-e4e8-44f6-a21b-9b83f31ac013 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:07:04.084844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935e194-e4e8-44f6-a21b-9b83f31ac013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:04.086207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935e194-e4e8-44f6-a21b-9b83f31ac013 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:04.094988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 596/1189] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:07:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:07:04.102754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87203446-3812-4554-b5fa-c2199b68e34a None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:04.154483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:04.225216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87203446-3812-4554-b5fa-c2199b68e34a None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:04.261512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41ed6c-be3c-4dc8-9038-5dd2df441abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:04.262116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41ed6c-be3c-4dc8-9038-5dd2df441abd None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:04.262385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41ed6c-be3c-4dc8-9038-5dd2df441abd None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:04.262647 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41ed6c-be3c-4dc8-9038-5dd2df441abd None admin] PATH_INFO: `/v3/users/df283df53ce749c9a0cecfa677d27d21` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:04.263862 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41ed6c-be3c-4dc8-9038-5dd2df441abd None admin] RBAC: Authorizing `identity:get_user(user_id=df283df53ce749c9a0cecfa677d27d21)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:04.265317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41ed6c-be3c-4dc8-9038-5dd2df441abd None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:04.266964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 594/1190] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:07:04 2022] GET /identity/v3/users/df283df53ce749c9a0cecfa677d27d21 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:07:04.276269 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935e194-e4e8-44f6-a21b-9b83f31ac013 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:04.387804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935e194-e4e8-44f6-a21b-9b83f31ac013 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:04.423792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5b0b7-feb8-42b8-85e7-65e034d95986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:04.424414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5b0b7-feb8-42b8-85e7-65e034d95986 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:04.424719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5b0b7-feb8-42b8-85e7-65e034d95986 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:04.424983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5b0b7-feb8-42b8-85e7-65e034d95986 None admin] PATH_INFO: `/v3/projects/b3b5ae30bafe47d1ac27e36b9f29f3ad` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:04.426236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5b0b7-feb8-42b8-85e7-65e034d95986 None admin] RBAC: Authorizing `identity:get_project(project_id=b3b5ae30bafe47d1ac27e36b9f29f3ad)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:04.427655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5b0b7-feb8-42b8-85e7-65e034d95986 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:04.429289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 597/1191] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:07:04 2022] GET /identity/v3/projects/b3b5ae30bafe47d1ac27e36b9f29f3ad => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:07:04.436644 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41ed6c-be3c-4dc8-9038-5dd2df441abd None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:04.554287 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41ed6c-be3c-4dc8-9038-5dd2df441abd None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:04.567560 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:04.594029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf035b-f1bb-4cb8-a3ea-4937decf5689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:04.594697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf035b-f1bb-4cb8-a3ea-4937decf5689 None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:04.594974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf035b-f1bb-4cb8-a3ea-4937decf5689 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:04.595241 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf035b-f1bb-4cb8-a3ea-4937decf5689 None admin] PATH_INFO: `/v3/projects/b3b5ae30bafe47d1ac27e36b9f29f3ad/users/df283df53ce749c9a0cecfa677d27d21/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:04.597267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf035b-f1bb-4cb8-a3ea-4937decf5689 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3b5ae30bafe47d1ac27e36b9f29f3ad, user_id=df283df53ce749c9a0cecfa677d27d21, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:04.598706 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf035b-f1bb-4cb8-a3ea-4937decf5689 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:04.613163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 595/1192] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:07:04 2022] PUT /identity/v3/projects/b3b5ae30bafe47d1ac27e36b9f29f3ad/users/df283df53ce749c9a0cecfa677d27d21/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:07:06.718237 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50024819-0cf4-47a5-a0b6-9a493f88e23e None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:06.719069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50024819-0cf4-47a5-a0b6-9a493f88e23e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:06.719069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50024819-0cf4-47a5-a0b6-9a493f88e23e None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:06.721350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 598/1193] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:07:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:07:06.727762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8217c-a644-426f-b60b-e9164a1f339f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:06.728478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8217c-a644-426f-b60b-e9164a1f339f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:06.728662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8217c-a644-426f-b60b-e9164a1f339f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:06.754650 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-67c8217c-a644-426f-b60b-e9164a1f339f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:07:06.755829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-67c8217c-a644-426f-b60b-e9164a1f339f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:07:06.828450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 596/1194] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:07:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:07:07.098681 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50024819-0cf4-47a5-a0b6-9a493f88e23e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:07.166181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:07.222157 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50024819-0cf4-47a5-a0b6-9a493f88e23e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:07.258738 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4de411-8101-488d-8454-c2e4c07522f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:07.259368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4de411-8101-488d-8454-c2e4c07522f5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:07.259635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4de411-8101-488d-8454-c2e4c07522f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:07.259896 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4de411-8101-488d-8454-c2e4c07522f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:07.270724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4de411-8101-488d-8454-c2e4c07522f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-7b4de411-8101-488d-8454-c2e4c07522f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:07:07.275450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:07:07.276500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:07:07.277391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:07:07.277391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:07:07.277391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:07:07.277391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:07:07.277391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:07:07.277391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:07:07.277391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:07:07.277391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:07:07.277391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:07:07.277391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:07:07.277391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:07:07.277391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:07:07.280813 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 599/1195] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:07:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:07:07.283473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8217c-a644-426f-b60b-e9164a1f339f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:07.394517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:07.397839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8217c-a644-426f-b60b-e9164a1f339f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:07.437383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35fe92-284b-4e8f-8266-b199028626dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:07.437983 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35fe92-284b-4e8f-8266-b199028626dc None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:07.438255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35fe92-284b-4e8f-8266-b199028626dc None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:07.438510 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35fe92-284b-4e8f-8266-b199028626dc None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:07.439163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35fe92-284b-4e8f-8266-b199028626dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:07:07.439452 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35fe92-284b-4e8f-8266-b199028626dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:07.440843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35fe92-284b-4e8f-8266-b199028626dc None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:07.448909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 597/1196] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:07:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:07:07.455895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4de411-8101-488d-8454-c2e4c07522f5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:07.566843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4de411-8101-488d-8454-c2e4c07522f5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:07.577559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:07.601428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607bacda-6b29-4ab7-ad0d-34d92c0ee7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:07.602048 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bacda-6b29-4ab7-ad0d-34d92c0ee7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:07.602319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bacda-6b29-4ab7-ad0d-34d92c0ee7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:07.602575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bacda-6b29-4ab7-ad0d-34d92c0ee7a8 None admin] PATH_INFO: `/v3/users/df283df53ce749c9a0cecfa677d27d21` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:07.603713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607bacda-6b29-4ab7-ad0d-34d92c0ee7a8 None admin] RBAC: Authorizing `identity:get_user(user_id=df283df53ce749c9a0cecfa677d27d21)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:07.605169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607bacda-6b29-4ab7-ad0d-34d92c0ee7a8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:07.606769 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 600/1197] 10.222.0.75 () {56 vars in 1177 bytes} [Fri Sep 23 15:07:07 2022] GET /identity/v3/users/df283df53ce749c9a0cecfa677d27d21 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:07:07.613253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35fe92-284b-4e8f-8266-b199028626dc None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:07.717906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35fe92-284b-4e8f-8266-b199028626dc None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:07.755672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad438a8-5b1f-46d6-858e-c5d19f74a2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:07.756318 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad438a8-5b1f-46d6-858e-c5d19f74a2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:07.756541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad438a8-5b1f-46d6-858e-c5d19f74a2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:07.756839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad438a8-5b1f-46d6-858e-c5d19f74a2f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:07.789255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:07.842072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad438a8-5b1f-46d6-858e-c5d19f74a2f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:07.843774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad438a8-5b1f-46d6-858e-c5d19f74a2f3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:07.959834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 598/1198] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:07:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:07:07.966127 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607bacda-6b29-4ab7-ad0d-34d92c0ee7a8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:08.067860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607bacda-6b29-4ab7-ad0d-34d92c0ee7a8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:08.104835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea75b8fa-a5d6-44e5-a1d1-d87ef0e98bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:08.105434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75b8fa-a5d6-44e5-a1d1-d87ef0e98bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:08.105698 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75b8fa-a5d6-44e5-a1d1-d87ef0e98bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:08.105962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea75b8fa-a5d6-44e5-a1d1-d87ef0e98bb4 None admin] PATH_INFO: `/v3/projects/b3b5ae30bafe47d1ac27e36b9f29f3ad` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:08.107138 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75b8fa-a5d6-44e5-a1d1-d87ef0e98bb4 None admin] RBAC: Authorizing `identity:get_project(project_id=b3b5ae30bafe47d1ac27e36b9f29f3ad)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:08.108541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea75b8fa-a5d6-44e5-a1d1-d87ef0e98bb4 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:08.110484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 601/1199] 10.222.0.75 () {56 vars in 1186 bytes} [Fri Sep 23 15:07:07 2022] GET /identity/v3/projects/b3b5ae30bafe47d1ac27e36b9f29f3ad => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:07:08.116849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad438a8-5b1f-46d6-858e-c5d19f74a2f3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:07:08.193559 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:08.237070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad438a8-5b1f-46d6-858e-c5d19f74a2f3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:07:08.275140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da277400-8cfd-4804-b1bf-96129868a05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:07:08.275730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da277400-8cfd-4804-b1bf-96129868a05f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:07:08.275998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da277400-8cfd-4804-b1bf-96129868a05f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:07:08.276263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da277400-8cfd-4804-b1bf-96129868a05f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:07:08.277015 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da277400-8cfd-4804-b1bf-96129868a05f None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=b3b5ae30bafe47d1ac27e36b9f29f3ad, user.id=df283df53ce749c9a0cecfa677d27d21) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:07:08.277322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da277400-8cfd-4804-b1bf-96129868a05f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:07:08.278773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da277400-8cfd-4804-b1bf-96129868a05f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:07:08.284822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 599/1200] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:07:08 2022] GET /identity/v3/role_assignments?user.id=df283df53ce749c9a0cecfa677d27d21&scope.project.id=b3b5ae30bafe47d1ac27e36b9f29f3ad&role.id=ae444a489623434999d758b71ae2dc87 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:07:18.100134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=77 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:07:18.282185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=37 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:04.681180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbaeb59-8e80-4615-a4a4-2da1e4ed6cfc None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:04.681779 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbaeb59-8e80-4615-a4a4-2da1e4ed6cfc None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:04.681779 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbaeb59-8e80-4615-a4a4-2da1e4ed6cfc None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:04.683212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 602/1201] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:08:04.690687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf5a7a-3c00-44bb-bfee-f507a79c3832 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:04.691266 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf5a7a-3c00-44bb-bfee-f507a79c3832 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:04.691266 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf5a7a-3c00-44bb-bfee-f507a79c3832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:04.711631 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-93cf5a7a-3c00-44bb-bfee-f507a79c3832 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:08:04.713027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-93cf5a7a-3c00-44bb-bfee-f507a79c3832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:08:04.803393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 600/1202] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:08:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:08:05.103149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbaeb59-8e80-4615-a4a4-2da1e4ed6cfc None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:05.244999 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbaeb59-8e80-4615-a4a4-2da1e4ed6cfc None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:05.299724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbb951-47d9-4604-ab65-aec8c4c69379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:05.300374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbb951-47d9-4604-ab65-aec8c4c69379 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:05.300671 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbb951-47d9-4604-ab65-aec8c4c69379 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:05.300939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bbb951-47d9-4604-ab65-aec8c4c69379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:05.316084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bbb951-47d9-4604-ab65-aec8c4c69379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:05.317573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bbb951-47d9-4604-ab65-aec8c4c69379 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-d0bbb951-47d9-4604-ab65-aec8c4c69379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:08:05.333631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:05.336839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:08:05.338057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:08:05.338057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:05.338057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:05.338057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 603/1203] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:05.341116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf5a7a-3c00-44bb-bfee-f507a79c3832 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:05.456499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf5a7a-3c00-44bb-bfee-f507a79c3832 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:05.493331 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:05.499687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5e70d-1eae-4e37-b34a-da69fe9bf26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:05.500682 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5e70d-1eae-4e37-b34a-da69fe9bf26b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:05.500988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5e70d-1eae-4e37-b34a-da69fe9bf26b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:05.501497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5e70d-1eae-4e37-b34a-da69fe9bf26b None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:05.502210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5e70d-1eae-4e37-b34a-da69fe9bf26b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:05.502512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5e70d-1eae-4e37-b34a-da69fe9bf26b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:05.503995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5e70d-1eae-4e37-b34a-da69fe9bf26b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:05.517605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 601/1204] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:05.523800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbb951-47d9-4604-ab65-aec8c4c69379 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:05.597996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:05.654756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bbb951-47d9-4604-ab65-aec8c4c69379 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:05.693672 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700cea5-6bf8-4e1a-afc8-4cdfda3c72a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:05.694654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700cea5-6bf8-4e1a-afc8-4cdfda3c72a0 None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:05.695256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700cea5-6bf8-4e1a-afc8-4cdfda3c72a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:05.695256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700cea5-6bf8-4e1a-afc8-4cdfda3c72a0 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:05.696317 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e700cea5-6bf8-4e1a-afc8-4cdfda3c72a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:05.697582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e700cea5-6bf8-4e1a-afc8-4cdfda3c72a0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:05.735062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 604/1205] 10.222.0.75 () {60 vars in 1132 bytes} [Fri Sep 23 15:08:05 2022] POST /identity/v3/users => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:08:08.151907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ac96c-3aeb-4773-8762-a6ec2f72de50 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:08.152505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ac96c-3aeb-4773-8762-a6ec2f72de50 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:08.152505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ac96c-3aeb-4773-8762-a6ec2f72de50 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:08.154050 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 602/1206] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:08:08.161143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d082ad-dd1c-4a86-8e71-2a17208adaac None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:08.161763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d082ad-dd1c-4a86-8e71-2a17208adaac None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:08.161880 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d082ad-dd1c-4a86-8e71-2a17208adaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:08.183786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d4d082ad-dd1c-4a86-8e71-2a17208adaac None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:08:08.185094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d4d082ad-dd1c-4a86-8e71-2a17208adaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:08:08.260421 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 605/1207] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:08:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:08:08.526029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ac96c-3aeb-4773-8762-a6ec2f72de50 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:08.645564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ac96c-3aeb-4773-8762-a6ec2f72de50 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:08.684451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e1cd2-6e52-42b4-88b5-b0a5a89865c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:08.685499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e1cd2-6e52-42b4-88b5-b0a5a89865c1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:08.685876 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e1cd2-6e52-42b4-88b5-b0a5a89865c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:08.686156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e1cd2-6e52-42b4-88b5-b0a5a89865c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:08.698206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e1cd2-6e52-42b4-88b5-b0a5a89865c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-791e1cd2-6e52-42b4-88b5-b0a5a89865c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:08:08.703270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:08.704357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:08.706616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:08.706616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:08.706616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:08.706616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:08.706616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:08:08.706616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:08:08.706616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:08:08.706616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:08:08.706616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:08:08.706616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:08:08.706616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:08:08.706616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:08.706616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 603/1208] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:08:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:08.711288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d082ad-dd1c-4a86-8e71-2a17208adaac None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:08.755156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:08.826621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d082ad-dd1c-4a86-8e71-2a17208adaac None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:08.868498 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6b7e3-9889-4517-bf4d-98bc11bf56a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:08.869168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6b7e3-9889-4517-bf4d-98bc11bf56a8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:08.869446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6b7e3-9889-4517-bf4d-98bc11bf56a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:08.869740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6b7e3-9889-4517-bf4d-98bc11bf56a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:08.870405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6b7e3-9889-4517-bf4d-98bc11bf56a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:08.870758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6b7e3-9889-4517-bf4d-98bc11bf56a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:08.872099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6b7e3-9889-4517-bf4d-98bc11bf56a8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:08.881214 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 606/1209] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:08:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:08.887528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e1cd2-6e52-42b4-88b5-b0a5a89865c1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:08.940120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:09.001429 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e1cd2-6e52-42b4-88b5-b0a5a89865c1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:09.041489 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69232b26-c6b9-4c80-8680-849b70b0ceda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:09.042184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69232b26-c6b9-4c80-8680-849b70b0ceda None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:09.042564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69232b26-c6b9-4c80-8680-849b70b0ceda None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:09.042879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69232b26-c6b9-4c80-8680-849b70b0ceda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:09.056142 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69232b26-c6b9-4c80-8680-849b70b0ceda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:09.057651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69232b26-c6b9-4c80-8680-849b70b0ceda None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-69232b26-c6b9-4c80-8680-849b70b0ceda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:08:09.074796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:09.076774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:08:09.077904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:08:09.077904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:09.077904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:09.080793 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 604/1210] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:09.083053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6b7e3-9889-4517-bf4d-98bc11bf56a8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:09.196708 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:09.200523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6b7e3-9889-4517-bf4d-98bc11bf56a8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:09.236881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e31ef-940d-4e55-a629-71514446bdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:09.237561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e31ef-940d-4e55-a629-71514446bdbf None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:09.237940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e31ef-940d-4e55-a629-71514446bdbf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:09.238305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e31ef-940d-4e55-a629-71514446bdbf None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:09.239139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e31ef-940d-4e55-a629-71514446bdbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:09.239530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e31ef-940d-4e55-a629-71514446bdbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:09.241445 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e31ef-940d-4e55-a629-71514446bdbf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:09.258008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 607/1211] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:09.265887 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69232b26-c6b9-4c80-8680-849b70b0ceda None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:09.393133 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69232b26-c6b9-4c80-8680-849b70b0ceda None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:09.395058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:09.431539 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766da382-567b-4257-8324-53d1002ca916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:09.432164 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766da382-567b-4257-8324-53d1002ca916 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:09.432438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766da382-567b-4257-8324-53d1002ca916 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:09.432729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766da382-567b-4257-8324-53d1002ca916 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:09.443399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766da382-567b-4257-8324-53d1002ca916 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:09.445030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766da382-567b-4257-8324-53d1002ca916 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-766da382-567b-4257-8324-53d1002ca916 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:08:09.458749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:09.460038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:09.461109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:09.461109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:09.461109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:08:09.461109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:08:09.461109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:08:09.461109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:08:09.461109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:08:09.461109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:08:09.461109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:08:09.461109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:09.461109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 605/1212] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:09.468866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e31ef-940d-4e55-a629-71514446bdbf None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:09.583927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e31ef-940d-4e55-a629-71514446bdbf None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:09.619533 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee45e97-fc73-4bf4-8e4c-46c9b6c8ede1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:09.620138 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee45e97-fc73-4bf4-8e4c-46c9b6c8ede1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:09.620422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee45e97-fc73-4bf4-8e4c-46c9b6c8ede1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:09.620718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee45e97-fc73-4bf4-8e4c-46c9b6c8ede1 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:09.621438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee45e97-fc73-4bf4-8e4c-46c9b6c8ede1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:09.621731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee45e97-fc73-4bf4-8e4c-46c9b6c8ede1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:09.623142 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee45e97-fc73-4bf4-8e4c-46c9b6c8ede1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:09.645099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 608/1213] 10.222.0.75 () {56 vars in 1143 bytes} [Fri Sep 23 15:08:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:09.646387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766da382-567b-4257-8324-53d1002ca916 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:09.754740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766da382-567b-4257-8324-53d1002ca916 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:09.789870 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0391a8-fe02-45c8-a56e-472d9c003f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:09.790675 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0391a8-fe02-45c8-a56e-472d9c003f41 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:09.791044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0391a8-fe02-45c8-a56e-472d9c003f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:09.791431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0391a8-fe02-45c8-a56e-472d9c003f41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:09.823293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:09.872503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0391a8-fe02-45c8-a56e-472d9c003f41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:09.874001 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0391a8-fe02-45c8-a56e-472d9c003f41 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:09.990142 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 606/1214] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:08:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:08:09.995866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee45e97-fc73-4bf4-8e4c-46c9b6c8ede1 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:10.021482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:10.115475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee45e97-fc73-4bf4-8e4c-46c9b6c8ede1 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:10.158687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767faea2-fe5b-4613-bb8f-bfb9a04f1b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:10.159346 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767faea2-fe5b-4613-bb8f-bfb9a04f1b44 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:10.159622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767faea2-fe5b-4613-bb8f-bfb9a04f1b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:10.159889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767faea2-fe5b-4613-bb8f-bfb9a04f1b44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:10.171679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767faea2-fe5b-4613-bb8f-bfb9a04f1b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:10.173815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767faea2-fe5b-4613-bb8f-bfb9a04f1b44 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-767faea2-fe5b-4613-bb8f-bfb9a04f1b44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:08:10.195670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:10.197480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:08:10.204089 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:08:10.204089 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:10.204089 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:10.204089 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 609/1215] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:10.204657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0391a8-fe02-45c8-a56e-472d9c003f41 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:10.272977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:10.315466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0391a8-fe02-45c8-a56e-472d9c003f41 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:10.353537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca5475-6236-484d-9497-ce0f0f217104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:10.354117 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca5475-6236-484d-9497-ce0f0f217104 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:10.354385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca5475-6236-484d-9497-ce0f0f217104 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:10.354645 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca5475-6236-484d-9497-ce0f0f217104 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:10.355436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca5475-6236-484d-9497-ce0f0f217104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:10.355701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca5475-6236-484d-9497-ce0f0f217104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:10.357163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca5475-6236-484d-9497-ce0f0f217104 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:10.370107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 607/1216] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:10.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767faea2-fe5b-4613-bb8f-bfb9a04f1b44 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:10.460034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:10.498306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767faea2-fe5b-4613-bb8f-bfb9a04f1b44 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:10.543366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd1565-510b-4d95-a8ce-993b994e4386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:10.543941 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd1565-510b-4d95-a8ce-993b994e4386 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:10.544212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd1565-510b-4d95-a8ce-993b994e4386 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:10.544475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd1565-510b-4d95-a8ce-993b994e4386 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:10.552865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd1565-510b-4d95-a8ce-993b994e4386 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:10.554305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd1565-510b-4d95-a8ce-993b994e4386 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-8ddd1565-510b-4d95-a8ce-993b994e4386 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:08:10.567319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:10.568437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:10.569541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:08:10.569541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:08:10.569541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:08:10.569541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:08:10.569541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:08:10.569541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:08:10.569541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:08:10.569541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:10.569541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 610/1217] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:08:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:10.573325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca5475-6236-484d-9497-ce0f0f217104 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:10.691792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca5475-6236-484d-9497-ce0f0f217104 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:10.718502 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:10.732525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0480ea1-bdfe-4977-a1b0-df74eb9b47cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:10.733318 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480ea1-bdfe-4977-a1b0-df74eb9b47cb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:10.733680 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480ea1-bdfe-4977-a1b0-df74eb9b47cb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:10.735011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480ea1-bdfe-4977-a1b0-df74eb9b47cb None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:10.735011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0480ea1-bdfe-4977-a1b0-df74eb9b47cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:10.735287 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0480ea1-bdfe-4977-a1b0-df74eb9b47cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:10.737283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0480ea1-bdfe-4977-a1b0-df74eb9b47cb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:10.755830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 608/1218] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:08:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:10.757501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd1565-510b-4d95-a8ce-993b994e4386 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:10.874187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd1565-510b-4d95-a8ce-993b994e4386 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:10.906981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:10.914216 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9b39e-4ce3-45e9-a461-a13dbe57e794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:10.914883 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9b39e-4ce3-45e9-a461-a13dbe57e794 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:10.915146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9b39e-4ce3-45e9-a461-a13dbe57e794 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:10.915496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9b39e-4ce3-45e9-a461-a13dbe57e794 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:10.916226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9b39e-4ce3-45e9-a461-a13dbe57e794 None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=2b8be1ec4a7a45b88d84bd9c48fab407) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:10.916528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9b39e-4ce3-45e9-a461-a13dbe57e794 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:10.917976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9b39e-4ce3-45e9-a461-a13dbe57e794 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:10.925198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 611/1219] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:08:10 2022] GET /identity/v3/role_assignments?user.id=2b8be1ec4a7a45b88d84bd9c48fab407&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:08:13.098384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e4ad2-5946-4720-8327-20a987749837 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:13.098935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e4ad2-5946-4720-8327-20a987749837 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:13.099025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e4ad2-5946-4720-8327-20a987749837 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:13.100852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 609/1220] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:08:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:08:13.106084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e023d0-f366-4afe-aedd-35e95f6c60b0 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:13.106518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e023d0-f366-4afe-aedd-35e95f6c60b0 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:13.106709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e023d0-f366-4afe-aedd-35e95f6c60b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:13.128294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-46e023d0-f366-4afe-aedd-35e95f6c60b0 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:08:13.129593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-46e023d0-f366-4afe-aedd-35e95f6c60b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:08:13.211579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 612/1221] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:08:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:08:13.529964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e4ad2-5946-4720-8327-20a987749837 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:13.644177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e4ad2-5946-4720-8327-20a987749837 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:13.685474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9ddbd-1696-46d6-981a-ede73fcca3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:13.686115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9ddbd-1696-46d6-981a-ede73fcca3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:13.686480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9ddbd-1696-46d6-981a-ede73fcca3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:13.686932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9ddbd-1696-46d6-981a-ede73fcca3e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:13.698513 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9ddbd-1696-46d6-981a-ede73fcca3e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-14e9ddbd-1696-46d6-981a-ede73fcca3e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:08:13.704223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:13.705311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:13.706204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:13.706204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:13.706204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:13.706204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:13.706204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:08:13.706204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:08:13.706204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:08:13.706204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:08:13.706204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:08:13.706204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:08:13.706204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:08:13.706204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:13.706204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 610/1222] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:08:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:13.713276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e023d0-f366-4afe-aedd-35e95f6c60b0 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:13.825937 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e023d0-f366-4afe-aedd-35e95f6c60b0 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:13.850761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:13.863370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccb03a-47f2-4962-b224-b17d827a3030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:13.864105 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccb03a-47f2-4962-b224-b17d827a3030 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:13.864396 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccb03a-47f2-4962-b224-b17d827a3030 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:13.864699 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccb03a-47f2-4962-b224-b17d827a3030 None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:13.865354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccb03a-47f2-4962-b224-b17d827a3030 None admin] RBAC: Adding query filter params (name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:13.865649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccb03a-47f2-4962-b224-b17d827a3030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:13.867001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccb03a-47f2-4962-b224-b17d827a3030 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:13.876098 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 613/1223] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:08:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:13.881966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9ddbd-1696-46d6-981a-ede73fcca3e2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:13.930949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:13.998791 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9ddbd-1696-46d6-981a-ede73fcca3e2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:14.040383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2723e-3f22-4a6e-918d-b284426ad123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:14.041076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2723e-3f22-4a6e-918d-b284426ad123 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:14.041359 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2723e-3f22-4a6e-918d-b284426ad123 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:14.041621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2723e-3f22-4a6e-918d-b284426ad123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:14.053135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2723e-3f22-4a6e-918d-b284426ad123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:14.054587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2723e-3f22-4a6e-918d-b284426ad123 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-afe2723e-3f22-4a6e-918d-b284426ad123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:08:14.070303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:14.071305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:08:14.072251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:08:14.072251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:14.072251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:14.072251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 611/1224] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:14.077453 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccb03a-47f2-4962-b224-b17d827a3030 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:14.202188 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccb03a-47f2-4962-b224-b17d827a3030 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:14.243991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d422a-c49e-43b4-becb-f00e5d13ebbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:14.244743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d422a-c49e-43b4-becb-f00e5d13ebbc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:14.245030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d422a-c49e-43b4-becb-f00e5d13ebbc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:14.245303 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d422a-c49e-43b4-becb-f00e5d13ebbc None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:14.246026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d422a-c49e-43b4-becb-f00e5d13ebbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:14.246329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d422a-c49e-43b4-becb-f00e5d13ebbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:14.247823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d422a-c49e-43b4-becb-f00e5d13ebbc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:14.263652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 614/1225] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:14.270718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2723e-3f22-4a6e-918d-b284426ad123 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:14.374107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:14.383239 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2723e-3f22-4a6e-918d-b284426ad123 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:14.426836 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f64b3-14f1-4325-920d-6f2c4b32cb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:14.427478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f64b3-14f1-4325-920d-6f2c4b32cb9d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:14.427778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f64b3-14f1-4325-920d-6f2c4b32cb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:14.428075 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f64b3-14f1-4325-920d-6f2c4b32cb9d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:14.437705 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f64b3-14f1-4325-920d-6f2c4b32cb9d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:14.439229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f64b3-14f1-4325-920d-6f2c4b32cb9d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-958f64b3-14f1-4325-920d-6f2c4b32cb9d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:08:14.451172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:14.452576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:14.454119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:14.454119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:14.454119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:08:14.454119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:08:14.454119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:08:14.454119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:08:14.454119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:08:14.454119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:08:14.454119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:08:14.454119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:14.454119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 612/1226] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:14.459908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d422a-c49e-43b4-becb-f00e5d13ebbc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:14.512455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:14.570417 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d422a-c49e-43b4-becb-f00e5d13ebbc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:14.609879 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d977186-b197-473b-9325-e53fe016c8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:14.610602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977186-b197-473b-9325-e53fe016c8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:14.610948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977186-b197-473b-9325-e53fe016c8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:14.611227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977186-b197-473b-9325-e53fe016c8c0 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:14.612469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d977186-b197-473b-9325-e53fe016c8c0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:14.612811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d977186-b197-473b-9325-e53fe016c8c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:14.614276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d977186-b197-473b-9325-e53fe016c8c0 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:14.632773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 615/1227] 10.222.0.75 () {56 vars in 1143 bytes} [Fri Sep 23 15:08:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:14.639545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f64b3-14f1-4325-920d-6f2c4b32cb9d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:14.747508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f64b3-14f1-4325-920d-6f2c4b32cb9d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:14.784420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc8d69-fff1-4858-ad63-aab868bec073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:14.785179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc8d69-fff1-4858-ad63-aab868bec073 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:14.785500 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc8d69-fff1-4858-ad63-aab868bec073 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:14.785781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc8d69-fff1-4858-ad63-aab868bec073 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:14.798955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc8d69-fff1-4858-ad63-aab868bec073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:14.800413 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc8d69-fff1-4858-ad63-aab868bec073 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-a2fc8d69-fff1-4858-ad63-aab868bec073 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:08:14.815281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:14.816795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:08:14.817982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:08:14.817982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:14.817982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:14.817982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 613/1228] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:14.822648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d977186-b197-473b-9325-e53fe016c8c0 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:14.929361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d977186-b197-473b-9325-e53fe016c8c0 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:14.931714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:14.974729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c2775-ee5a-49f2-be5d-75f349ac928e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:14.975367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c2775-ee5a-49f2-be5d-75f349ac928e None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:14.975601 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c2775-ee5a-49f2-be5d-75f349ac928e None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:14.975863 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c2775-ee5a-49f2-be5d-75f349ac928e None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:14.976526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c2775-ee5a-49f2-be5d-75f349ac928e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:14.976832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c2775-ee5a-49f2-be5d-75f349ac928e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:14.978249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c2775-ee5a-49f2-be5d-75f349ac928e None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:14.991512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 616/1229] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:08:14.997641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc8d69-fff1-4858-ad63-aab868bec073 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:15.009049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:15.110090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc8d69-fff1-4858-ad63-aab868bec073 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:15.149931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902b6c3-7364-4653-8b81-4ee9e2ec1462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:15.150636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902b6c3-7364-4653-8b81-4ee9e2ec1462 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:15.151058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902b6c3-7364-4653-8b81-4ee9e2ec1462 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:15.151303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902b6c3-7364-4653-8b81-4ee9e2ec1462 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:15.159907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902b6c3-7364-4653-8b81-4ee9e2ec1462 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:15.161594 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902b6c3-7364-4653-8b81-4ee9e2ec1462 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-1902b6c3-7364-4653-8b81-4ee9e2ec1462 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:08:15.178206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:15.179298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:15.180299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:08:15.180299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:08:15.180299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:08:15.180299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:08:15.180299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:08:15.180299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:08:15.180299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:08:15.180299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:15.180299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 614/1230] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:08:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:15.186048 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c2775-ee5a-49f2-be5d-75f349ac928e None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:15.301401 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c2775-ee5a-49f2-be5d-75f349ac928e None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:15.338886 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07b72a-4b8f-4a6e-b8c4-aac6487b80f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:15.339495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07b72a-4b8f-4a6e-b8c4-aac6487b80f1 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:15.339767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07b72a-4b8f-4a6e-b8c4-aac6487b80f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:15.340032 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07b72a-4b8f-4a6e-b8c4-aac6487b80f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:15.340750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07b72a-4b8f-4a6e-b8c4-aac6487b80f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:15.341049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07b72a-4b8f-4a6e-b8c4-aac6487b80f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:15.342466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07b72a-4b8f-4a6e-b8c4-aac6487b80f1 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:15.354694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 617/1231] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:08:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:15.360878 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902b6c3-7364-4653-8b81-4ee9e2ec1462 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:15.433919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:15.470748 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902b6c3-7364-4653-8b81-4ee9e2ec1462 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:15.506824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc7448-5494-496c-8311-4a64042a665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:15.507483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc7448-5494-496c-8311-4a64042a665a None admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:15.507740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc7448-5494-496c-8311-4a64042a665a None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:15.508007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc7448-5494-496c-8311-4a64042a665a None admin] PATH_INFO: `/v3/projects/27ac51280046479e82faaf3c0b550f72/users/2b8be1ec4a7a45b88d84bd9c48fab407/roles/351badda50da4e45ac1cb31482331c06` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:15.522307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc7448-5494-496c-8311-4a64042a665a None admin] RBAC: Authorizing `identity:create_grant(project_id=27ac51280046479e82faaf3c0b550f72, user_id=2b8be1ec4a7a45b88d84bd9c48fab407, role_id=351badda50da4e45ac1cb31482331c06)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:15.523727 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc7448-5494-496c-8311-4a64042a665a None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:15.538076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 615/1232] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:08:15 2022] PUT /identity/v3/projects/27ac51280046479e82faaf3c0b550f72/users/2b8be1ec4a7a45b88d84bd9c48fab407/roles/351badda50da4e45ac1cb31482331c06 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:08:17.786008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c4fff-6ef0-4408-b827-b3af8c9c5c02 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:17.786580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c4fff-6ef0-4408-b827-b3af8c9c5c02 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:17.786580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c4fff-6ef0-4408-b827-b3af8c9c5c02 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:17.787895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 618/1233] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:08:17.793328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19031a31-1f9f-4ca8-ae0d-78785c4d6805 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:17.793788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19031a31-1f9f-4ca8-ae0d-78785c4d6805 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:17.793886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19031a31-1f9f-4ca8-ae0d-78785c4d6805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:17.812192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-19031a31-1f9f-4ca8-ae0d-78785c4d6805 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:08:17.813410 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-19031a31-1f9f-4ca8-ae0d-78785c4d6805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:08:17.882308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 616/1234] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:08:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:08:18.141276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c4fff-6ef0-4408-b827-b3af8c9c5c02 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:18.174912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:18.262827 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c4fff-6ef0-4408-b827-b3af8c9c5c02 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:18.299607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80b018-75f5-4930-869a-9f83b19066a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:18.300541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80b018-75f5-4930-869a-9f83b19066a7 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:18.300824 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80b018-75f5-4930-869a-9f83b19066a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:18.301086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80b018-75f5-4930-869a-9f83b19066a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:18.312576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80b018-75f5-4930-869a-9f83b19066a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-6f80b018-75f5-4930-869a-9f83b19066a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:08:18.319061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:18.320702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:18.322548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:18.322548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:18.322548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:18.322548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:18.322548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:08:18.322548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:08:18.322548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:08:18.322548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:08:18.322548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:08:18.322548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:08:18.322548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 15:08:18.322548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:18.322548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 619/1235] 10.222.0.75 () {56 vars in 1102 bytes} [Fri Sep 23 15:08:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:18.328950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19031a31-1f9f-4ca8-ae0d-78785c4d6805 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:18.384542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:18.442247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19031a31-1f9f-4ca8-ae0d-78785c4d6805 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:18.482358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf4a3f-567c-4b8b-a5b8-c38ce7d354f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:18.482978 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf4a3f-567c-4b8b-a5b8-c38ce7d354f8 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:18.483301 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf4a3f-567c-4b8b-a5b8-c38ce7d354f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:18.483526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf4a3f-567c-4b8b-a5b8-c38ce7d354f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:18.484824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf4a3f-567c-4b8b-a5b8-c38ce7d354f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:18.485121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf4a3f-567c-4b8b-a5b8-c38ce7d354f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:18.486853 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf4a3f-567c-4b8b-a5b8-c38ce7d354f8 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:18.495932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 617/1236] 10.222.0.75 () {56 vars in 1103 bytes} [Fri Sep 23 15:08:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:18.502568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80b018-75f5-4930-869a-9f83b19066a7 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:18.595242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:18.612950 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80b018-75f5-4930-869a-9f83b19066a7 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:18.648311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87068969-2ec3-4735-a3c6-753edc5dd7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:18.649007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87068969-2ec3-4735-a3c6-753edc5dd7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:18.649328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87068969-2ec3-4735-a3c6-753edc5dd7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:18.649594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87068969-2ec3-4735-a3c6-753edc5dd7a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:18.660843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87068969-2ec3-4735-a3c6-753edc5dd7a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:18.662318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87068969-2ec3-4735-a3c6-753edc5dd7a8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-87068969-2ec3-4735-a3c6-753edc5dd7a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:08:18.679530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:18.685502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:08:18.686736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:08:18.686736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:18.686736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:18.686736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 620/1237] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:18.688275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf4a3f-567c-4b8b-a5b8-c38ce7d354f8 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:18.798665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf4a3f-567c-4b8b-a5b8-c38ce7d354f8 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:18.809805 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:18.837384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a91b37-2022-4224-8c8e-a93a56fdb3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:18.838751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a91b37-2022-4224-8c8e-a93a56fdb3aa None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:18.839254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a91b37-2022-4224-8c8e-a93a56fdb3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:18.839529 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a91b37-2022-4224-8c8e-a93a56fdb3aa None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:18.840316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a91b37-2022-4224-8c8e-a93a56fdb3aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:18.840733 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a91b37-2022-4224-8c8e-a93a56fdb3aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:18.842336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a91b37-2022-4224-8c8e-a93a56fdb3aa None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:18.855041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 618/1238] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:18.861097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87068969-2ec3-4735-a3c6-753edc5dd7a8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:18.964309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87068969-2ec3-4735-a3c6-753edc5dd7a8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:19.001638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd10e3-11af-4c4e-835e-8cf8a1fe1061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:19.002261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd10e3-11af-4c4e-835e-8cf8a1fe1061 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:19.002534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd10e3-11af-4c4e-835e-8cf8a1fe1061 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:19.002794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd10e3-11af-4c4e-835e-8cf8a1fe1061 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:19.005848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:19.011686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd10e3-11af-4c4e-835e-8cf8a1fe1061 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:19.013164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd10e3-11af-4c4e-835e-8cf8a1fe1061 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-8dcd10e3-11af-4c4e-835e-8cf8a1fe1061 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:08:19.024629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:19.025786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:19.027306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:19.027306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:19.027306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:08:19.027306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:08:19.027306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:08:19.027306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:08:19.027306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:08:19.027306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:08:19.027306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:08:19.027306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:19.027306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 621/1239] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:08:19.031889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a91b37-2022-4224-8c8e-a93a56fdb3aa None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:19.141308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a91b37-2022-4224-8c8e-a93a56fdb3aa None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:19.180446 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960517f-bef6-4530-89d8-38fc28c688d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:19.181373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960517f-bef6-4530-89d8-38fc28c688d2 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:19.181695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960517f-bef6-4530-89d8-38fc28c688d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:19.182100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960517f-bef6-4530-89d8-38fc28c688d2 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:19.183089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960517f-bef6-4530-89d8-38fc28c688d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:19.183431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960517f-bef6-4530-89d8-38fc28c688d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:19.185660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960517f-bef6-4530-89d8-38fc28c688d2 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:19.208885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 619/1240] 10.222.0.75 () {56 vars in 1143 bytes} [Fri Sep 23 15:08:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:19.215689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd10e3-11af-4c4e-835e-8cf8a1fe1061 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:19.323896 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd10e3-11af-4c4e-835e-8cf8a1fe1061 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:19.358720 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad4c9e-8872-4a99-82fc-9af611910a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:19.359405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad4c9e-8872-4a99-82fc-9af611910a4f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:19.359674 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad4c9e-8872-4a99-82fc-9af611910a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:19.359954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad4c9e-8872-4a99-82fc-9af611910a4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:19.438065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:19.445814 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad4c9e-8872-4a99-82fc-9af611910a4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:19.447551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad4c9e-8872-4a99-82fc-9af611910a4f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:19.559846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 622/1241] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:08:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:08:19.566094 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960517f-bef6-4530-89d8-38fc28c688d2 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:19.593718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:19.675355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960517f-bef6-4530-89d8-38fc28c688d2 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:19.715775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a1285-8d93-49f4-ac81-d632db1bffe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:19.716326 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a1285-8d93-49f4-ac81-d632db1bffe6 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:19.716617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a1285-8d93-49f4-ac81-d632db1bffe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:19.717029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a1285-8d93-49f4-ac81-d632db1bffe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:19.727449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a1285-8d93-49f4-ac81-d632db1bffe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:19.729259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a1285-8d93-49f4-ac81-d632db1bffe6 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-8d3a1285-8d93-49f4-ac81-d632db1bffe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:08:19.747833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:19.749035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:08:19.749969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:08:19.749969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:19.749969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:19.749969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 620/1242] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:19.755983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad4c9e-8872-4a99-82fc-9af611910a4f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:19.850706 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:19.857419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad4c9e-8872-4a99-82fc-9af611910a4f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:19.894108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bad23-4b28-42e9-aad1-67bde3f47dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:19.894780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bad23-4b28-42e9-aad1-67bde3f47dbb None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:19.895042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bad23-4b28-42e9-aad1-67bde3f47dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:19.895240 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bad23-4b28-42e9-aad1-67bde3f47dbb None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:19.896037 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bad23-4b28-42e9-aad1-67bde3f47dbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:19.896340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bad23-4b28-42e9-aad1-67bde3f47dbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:19.897732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bad23-4b28-42e9-aad1-67bde3f47dbb None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:19.909997 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 623/1243] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:19.917135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a1285-8d93-49f4-ac81-d632db1bffe6 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:20.006232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:20.030970 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a1285-8d93-49f4-ac81-d632db1bffe6 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:20.066767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f9ac8-3cf1-494e-89bf-b2265947c34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:20.067399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f9ac8-3cf1-494e-89bf-b2265947c34f None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:20.067672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f9ac8-3cf1-494e-89bf-b2265947c34f None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:20.067939 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f9ac8-3cf1-494e-89bf-b2265947c34f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:20.075027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f9ac8-3cf1-494e-89bf-b2265947c34f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:20.076689 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f9ac8-3cf1-494e-89bf-b2265947c34f None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-566f9ac8-3cf1-494e-89bf-b2265947c34f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:08:20.090017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:20.092090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:20.093566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:08:20.093566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:08:20.093566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:08:20.093566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:08:20.093566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:08:20.093566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:08:20.093566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:08:20.093566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:20.093566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 621/1244] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:08:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:20.098450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bad23-4b28-42e9-aad1-67bde3f47dbb None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:20.215313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bad23-4b28-42e9-aad1-67bde3f47dbb None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:20.251621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a1e9e-a03f-442e-b49d-833c7958249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:20.252231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a1e9e-a03f-442e-b49d-833c7958249a None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:20.252509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a1e9e-a03f-442e-b49d-833c7958249a None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:20.252801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a1e9e-a03f-442e-b49d-833c7958249a None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:20.253513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a1e9e-a03f-442e-b49d-833c7958249a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:20.253800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a1e9e-a03f-442e-b49d-833c7958249a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:20.255223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a1e9e-a03f-442e-b49d-833c7958249a None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:20.268083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 624/1245] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:08:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:20.274363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f9ac8-3cf1-494e-89bf-b2265947c34f None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:20.384730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f9ac8-3cf1-494e-89bf-b2265947c34f None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:20.422843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf64479-7f9f-4855-b4f1-8aaae5bde9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:20.423453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf64479-7f9f-4855-b4f1-8aaae5bde9cf None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:20.423719 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf64479-7f9f-4855-b4f1-8aaae5bde9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:20.423986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf64479-7f9f-4855-b4f1-8aaae5bde9cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:20.424758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf64479-7f9f-4855-b4f1-8aaae5bde9cf None admin] RBAC: Adding query filter params (role.id=351badda50da4e45ac1cb31482331c06, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=2b8be1ec4a7a45b88d84bd9c48fab407) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:20.425105 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf64479-7f9f-4855-b4f1-8aaae5bde9cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:20.426594 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf64479-7f9f-4855-b4f1-8aaae5bde9cf None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:20.433699 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 622/1246] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:08:20 2022] GET /identity/v3/role_assignments?user.id=2b8be1ec4a7a45b88d84bd9c48fab407&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=351badda50da4e45ac1cb31482331c06 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:22.749781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf05fd8-8ecc-4468-8cfc-6a3efc26cd4a None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:22.750406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf05fd8-8ecc-4468-8cfc-6a3efc26cd4a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:22.750406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf05fd8-8ecc-4468-8cfc-6a3efc26cd4a None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:22.751763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 625/1247] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:08:22.757215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f906e0d-1253-4725-a287-9a3c947f9c05 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:22.757735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f906e0d-1253-4725-a287-9a3c947f9c05 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:22.757735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f906e0d-1253-4725-a287-9a3c947f9c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:22.764832 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:22.780309 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7f906e0d-1253-4725-a287-9a3c947f9c05 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:08:22.781740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7f906e0d-1253-4725-a287-9a3c947f9c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:08:22.857751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 623/1248] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:08:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:08:23.138615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf05fd8-8ecc-4468-8cfc-6a3efc26cd4a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:23.158243 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:23.257583 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf05fd8-8ecc-4468-8cfc-6a3efc26cd4a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:23.294754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d1b28-8f1a-4b0d-9b2a-30d9b0702b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:23.295421 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d1b28-8f1a-4b0d-9b2a-30d9b0702b72 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:23.295683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d1b28-8f1a-4b0d-9b2a-30d9b0702b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:23.295982 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d1b28-8f1a-4b0d-9b2a-30d9b0702b72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:23.308239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d1b28-8f1a-4b0d-9b2a-30d9b0702b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-790d1b28-8f1a-4b0d-9b2a-30d9b0702b72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:08:23.313719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:23.315003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:23.316581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:23.316581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:23.316581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:23.316581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:23.316581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:08:23.316581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:08:23.316581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:08:23.316581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:08:23.316581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:08:23.316581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:08:23.316581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:08:23.316581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:23.316581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 626/1249] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:08:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:23.321601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f906e0d-1253-4725-a287-9a3c947f9c05 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:23.430915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f906e0d-1253-4725-a287-9a3c947f9c05 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:23.462291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:23.467735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf192a7-5619-47f6-86b9-c53d1d4fcf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:23.469005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf192a7-5619-47f6-86b9-c53d1d4fcf9d None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:23.469287 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf192a7-5619-47f6-86b9-c53d1d4fcf9d None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:23.469555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf192a7-5619-47f6-86b9-c53d1d4fcf9d None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:23.470229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf192a7-5619-47f6-86b9-c53d1d4fcf9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:23.470529 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf192a7-5619-47f6-86b9-c53d1d4fcf9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:23.471896 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf192a7-5619-47f6-86b9-c53d1d4fcf9d None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:23.480224 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 624/1250] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:08:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:23.487686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d1b28-8f1a-4b0d-9b2a-30d9b0702b72 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:23.572835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:23.599383 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d1b28-8f1a-4b0d-9b2a-30d9b0702b72 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:23.639530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189286de-eed1-4a7a-bb59-a08e8a1c5b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:23.640398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189286de-eed1-4a7a-bb59-a08e8a1c5b14 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:23.640765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189286de-eed1-4a7a-bb59-a08e8a1c5b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:23.641169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189286de-eed1-4a7a-bb59-a08e8a1c5b14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:23.653001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189286de-eed1-4a7a-bb59-a08e8a1c5b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:23.654459 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189286de-eed1-4a7a-bb59-a08e8a1c5b14 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-189286de-eed1-4a7a-bb59-a08e8a1c5b14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:08:23.669376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:23.670506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:08:23.672252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:08:23.672252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:23.672252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:23.672252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 627/1251] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:23.676728 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf192a7-5619-47f6-86b9-c53d1d4fcf9d None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:23.785599 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf192a7-5619-47f6-86b9-c53d1d4fcf9d None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:23.826975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac347a-79da-40a8-9a52-f8c01437e2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:23.827712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac347a-79da-40a8-9a52-f8c01437e2ac None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:23.828025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac347a-79da-40a8-9a52-f8c01437e2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:23.828446 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac347a-79da-40a8-9a52-f8c01437e2ac None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:23.828867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac347a-79da-40a8-9a52-f8c01437e2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:23.829152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac347a-79da-40a8-9a52-f8c01437e2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:23.830513 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac347a-79da-40a8-9a52-f8c01437e2ac None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:23.845308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 625/1252] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:08:23.851765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189286de-eed1-4a7a-bb59-a08e8a1c5b14 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:23.960983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189286de-eed1-4a7a-bb59-a08e8a1c5b14 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:23.989603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:23.997734 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2feacb-e45a-44a4-9a00-2be9ea03a1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:23.998342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2feacb-e45a-44a4-9a00-2be9ea03a1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:23.998630 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2feacb-e45a-44a4-9a00-2be9ea03a1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:23.998872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2feacb-e45a-44a4-9a00-2be9ea03a1f5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:24.006975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2feacb-e45a-44a4-9a00-2be9ea03a1f5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:24.008415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2feacb-e45a-44a4-9a00-2be9ea03a1f5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-aa2feacb-e45a-44a4-9a00-2be9ea03a1f5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:08:24.020287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:24.021557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:24.022725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:24.022725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:24.022725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:08:24.022725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:08:24.022725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:08:24.022725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:08:24.022725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:08:24.022725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:08:24.022725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:08:24.022725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:24.022725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 628/1253] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:24.028434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac347a-79da-40a8-9a52-f8c01437e2ac None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:24.090607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:24.141347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac347a-79da-40a8-9a52-f8c01437e2ac None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:24.183511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b5e1f-4dba-40f8-8d18-22986f8e0ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:24.184519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b5e1f-4dba-40f8-8d18-22986f8e0ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:24.184519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b5e1f-4dba-40f8-8d18-22986f8e0ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:24.184771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b5e1f-4dba-40f8-8d18-22986f8e0ee4 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:24.185445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b5e1f-4dba-40f8-8d18-22986f8e0ee4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:24.185750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b5e1f-4dba-40f8-8d18-22986f8e0ee4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:24.187302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b5e1f-4dba-40f8-8d18-22986f8e0ee4 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:24.208440 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 626/1254] 10.222.0.75 () {56 vars in 1143 bytes} [Fri Sep 23 15:08:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:24.214653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2feacb-e45a-44a4-9a00-2be9ea03a1f5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:24.334518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2feacb-e45a-44a4-9a00-2be9ea03a1f5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:24.370754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c289775-93e6-4fe8-b329-481bd3229b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:24.371317 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c289775-93e6-4fe8-b329-481bd3229b72 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:24.371584 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c289775-93e6-4fe8-b329-481bd3229b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:24.371846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c289775-93e6-4fe8-b329-481bd3229b72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:24.435574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:24.450388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c289775-93e6-4fe8-b329-481bd3229b72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:24.452061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c289775-93e6-4fe8-b329-481bd3229b72 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:24.565253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 629/1255] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:08:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:08:24.571459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b5e1f-4dba-40f8-8d18-22986f8e0ee4 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:24.687111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b5e1f-4dba-40f8-8d18-22986f8e0ee4 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:24.701612 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:24.725711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d668e80-4002-41c0-b233-537a94a28760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:24.726347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d668e80-4002-41c0-b233-537a94a28760 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:24.726613 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d668e80-4002-41c0-b233-537a94a28760 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:24.726872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d668e80-4002-41c0-b233-537a94a28760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:24.738573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d668e80-4002-41c0-b233-537a94a28760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:24.740132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d668e80-4002-41c0-b233-537a94a28760 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-4d668e80-4002-41c0-b233-537a94a28760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:08:24.756150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:24.757374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:08:24.758277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:08:24.758277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:24.758277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:24.758277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 627/1256] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:24.764096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c289775-93e6-4fe8-b329-481bd3229b72 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:24.857710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:24.872315 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c289775-93e6-4fe8-b329-481bd3229b72 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:24.907696 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9ee6c-8daa-40f7-aa12-cd797ed10ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:24.908296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9ee6c-8daa-40f7-aa12-cd797ed10ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:24.908575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9ee6c-8daa-40f7-aa12-cd797ed10ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:24.908898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9ee6c-8daa-40f7-aa12-cd797ed10ce5 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:24.909533 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9ee6c-8daa-40f7-aa12-cd797ed10ce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:24.909829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9ee6c-8daa-40f7-aa12-cd797ed10ce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:24.911176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9ee6c-8daa-40f7-aa12-cd797ed10ce5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:24.922568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 630/1257] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:24.928682 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d668e80-4002-41c0-b233-537a94a28760 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:25.046666 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d668e80-4002-41c0-b233-537a94a28760 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:25.083606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693c758-55ff-4cea-9ee3-e4ad78ddf086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:25.084293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693c758-55ff-4cea-9ee3-e4ad78ddf086 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:25.084543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693c758-55ff-4cea-9ee3-e4ad78ddf086 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:25.084871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693c758-55ff-4cea-9ee3-e4ad78ddf086 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:25.092155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693c758-55ff-4cea-9ee3-e4ad78ddf086 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:25.093682 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693c758-55ff-4cea-9ee3-e4ad78ddf086 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-5693c758-55ff-4cea-9ee3-e4ad78ddf086 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:08:25.105382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:25.106480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:25.107545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:08:25.107545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:08:25.107545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:08:25.107545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:08:25.107545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:08:25.107545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:08:25.107545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:08:25.107545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:25.107545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 628/1258] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:08:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:25.113506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9ee6c-8daa-40f7-aa12-cd797ed10ce5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:25.221210 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9ee6c-8daa-40f7-aa12-cd797ed10ce5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:25.253736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904bab0-5180-44ee-9788-779709690083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:25.254326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904bab0-5180-44ee-9788-779709690083 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:25.254607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904bab0-5180-44ee-9788-779709690083 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:25.254879 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904bab0-5180-44ee-9788-779709690083 None admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:25.255569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904bab0-5180-44ee-9788-779709690083 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:25.255863 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904bab0-5180-44ee-9788-779709690083 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:25.257338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904bab0-5180-44ee-9788-779709690083 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:25.270268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 631/1259] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:08:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:25.277626 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693c758-55ff-4cea-9ee3-e4ad78ddf086 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:25.309849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:25.398823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693c758-55ff-4cea-9ee3-e4ad78ddf086 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:25.435476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bfe0f-84c9-4a2d-8bf3-ae49fa0332a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:25.436070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bfe0f-84c9-4a2d-8bf3-ae49fa0332a7 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:25.436345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bfe0f-84c9-4a2d-8bf3-ae49fa0332a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:25.436631 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bfe0f-84c9-4a2d-8bf3-ae49fa0332a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:25.437490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bfe0f-84c9-4a2d-8bf3-ae49fa0332a7 None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=2b8be1ec4a7a45b88d84bd9c48fab407) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:25.437798 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bfe0f-84c9-4a2d-8bf3-ae49fa0332a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:25.439229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bfe0f-84c9-4a2d-8bf3-ae49fa0332a7 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:25.445274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 629/1260] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:08:25 2022] GET /identity/v3/role_assignments?user.id=2b8be1ec4a7a45b88d84bd9c48fab407&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=ae444a489623434999d758b71ae2dc87 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:27.587747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4af94-c266-40e5-b220-d5efb510827f None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:27.588280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4af94-c266-40e5-b220-d5efb510827f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:27.588280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4af94-c266-40e5-b220-d5efb510827f None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:27.589701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 632/1261] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:08:27.595905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7e906-5274-450f-923e-dd8b78e0607a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:27.596399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7e906-5274-450f-923e-dd8b78e0607a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:27.596399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7e906-5274-450f-923e-dd8b78e0607a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:27.616903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-cca7e906-5274-450f-923e-dd8b78e0607a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:08:27.618088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-cca7e906-5274-450f-923e-dd8b78e0607a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:08:27.689813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 630/1262] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:08:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 15:08:27.961541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4af94-c266-40e5-b220-d5efb510827f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:27.975095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:28.079763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4af94-c266-40e5-b220-d5efb510827f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:28.115061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b2f53-1b94-4655-a79d-90a61d6c2e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:28.115666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b2f53-1b94-4655-a79d-90a61d6c2e7f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:28.115939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b2f53-1b94-4655-a79d-90a61d6c2e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:28.116207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b2f53-1b94-4655-a79d-90a61d6c2e7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:28.128450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b2f53-1b94-4655-a79d-90a61d6c2e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-117b2f53-1b94-4655-a79d-90a61d6c2e7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:08:28.133306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:28.134366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:28.135325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:28.135325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:28.135325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:28.135325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:28.135325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:08:28.135325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:08:28.135325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:08:28.135325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:08:28.135325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:08:28.135325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:08:28.135325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:08:28.135325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:28.135325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 633/1263] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:08:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:28.140898 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7e906-5274-450f-923e-dd8b78e0607a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:28.161065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:28.252833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7e906-5274-450f-923e-dd8b78e0607a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:28.288996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31db5-7016-4a7a-bb04-ed57763124b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:28.289611 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31db5-7016-4a7a-bb04-ed57763124b3 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:28.289879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31db5-7016-4a7a-bb04-ed57763124b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:28.290144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef31db5-7016-4a7a-bb04-ed57763124b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:28.290829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef31db5-7016-4a7a-bb04-ed57763124b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:28.291127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef31db5-7016-4a7a-bb04-ed57763124b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:28.292629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef31db5-7016-4a7a-bb04-ed57763124b3 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:28.301205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 631/1264] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:08:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:28.307303 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b2f53-1b94-4655-a79d-90a61d6c2e7f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:28.387594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:28.422219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b2f53-1b94-4655-a79d-90a61d6c2e7f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:28.458491 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a3a1d-c86f-47fe-878d-5fe81b61d7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:28.459213 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a3a1d-c86f-47fe-878d-5fe81b61d7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:28.459486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a3a1d-c86f-47fe-878d-5fe81b61d7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:28.459749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a3a1d-c86f-47fe-878d-5fe81b61d7c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:28.471196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a3a1d-c86f-47fe-878d-5fe81b61d7c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:28.472652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a3a1d-c86f-47fe-878d-5fe81b61d7c5 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-9c1a3a1d-c86f-47fe-878d-5fe81b61d7c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:08:28.487715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:28.488940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:08:28.490006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:08:28.490006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:28.490006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:28.490006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 634/1265] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:28.495853 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31db5-7016-4a7a-bb04-ed57763124b3 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:28.587072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:28.607459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef31db5-7016-4a7a-bb04-ed57763124b3 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:28.643438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09741ce9-4a68-462a-99b7-871c2465bb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:28.644041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09741ce9-4a68-462a-99b7-871c2465bb64 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:28.644307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09741ce9-4a68-462a-99b7-871c2465bb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:28.644563 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09741ce9-4a68-462a-99b7-871c2465bb64 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:28.645257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09741ce9-4a68-462a-99b7-871c2465bb64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:28.645570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09741ce9-4a68-462a-99b7-871c2465bb64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:28.646972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09741ce9-4a68-462a-99b7-871c2465bb64 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:28.658654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 632/1266] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:28.665486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a3a1d-c86f-47fe-878d-5fe81b61d7c5 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:28.779051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a3a1d-c86f-47fe-878d-5fe81b61d7c5 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:28.801851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:28.819710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4d349-ab95-4e7f-8e60-1a822a8d13d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:28.820359 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d349-ab95-4e7f-8e60-1a822a8d13d8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:28.820628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d349-ab95-4e7f-8e60-1a822a8d13d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:28.820923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d349-ab95-4e7f-8e60-1a822a8d13d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:28.829528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4d349-ab95-4e7f-8e60-1a822a8d13d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:28.831151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4d349-ab95-4e7f-8e60-1a822a8d13d8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-1bd4d349-ab95-4e7f-8e60-1a822a8d13d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:08:28.842377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:28.843467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:28.845011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:28.845011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:28.845011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:08:28.845011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:08:28.845011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:08:28.845011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:08:28.845011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:08:28.845011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:08:28.845011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:08:28.845011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:28.845011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 635/1267] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:28.849944 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09741ce9-4a68-462a-99b7-871c2465bb64 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:28.962747 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09741ce9-4a68-462a-99b7-871c2465bb64 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:28.998591 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e576b2-c76b-41ee-802a-409a2210d016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:28.999186 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e576b2-c76b-41ee-802a-409a2210d016 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:28.999455 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e576b2-c76b-41ee-802a-409a2210d016 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:28.999740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e576b2-c76b-41ee-802a-409a2210d016 None admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:29.000488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e576b2-c76b-41ee-802a-409a2210d016 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:29.000815 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e576b2-c76b-41ee-802a-409a2210d016 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:29.002239 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e576b2-c76b-41ee-802a-409a2210d016 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:29.010880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:29.025138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 633/1268] 10.222.0.75 () {56 vars in 1143 bytes} [Fri Sep 23 15:08:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:29.031700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4d349-ab95-4e7f-8e60-1a822a8d13d8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:29.136949 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4d349-ab95-4e7f-8e60-1a822a8d13d8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:29.174650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951d3b8-ff72-4f17-ba62-7c28f14b7a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:29.175453 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951d3b8-ff72-4f17-ba62-7c28f14b7a5f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:29.175787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951d3b8-ff72-4f17-ba62-7c28f14b7a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:29.176090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951d3b8-ff72-4f17-ba62-7c28f14b7a5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:29.189300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951d3b8-ff72-4f17-ba62-7c28f14b7a5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:29.190749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951d3b8-ff72-4f17-ba62-7c28f14b7a5f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-3951d3b8-ff72-4f17-ba62-7c28f14b7a5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:08:29.205503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:29.206579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:08:29.207491 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:08:29.207491 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:29.207491 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:29.208760 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 636/1269] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:29.212545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e576b2-c76b-41ee-802a-409a2210d016 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:29.319953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e576b2-c76b-41ee-802a-409a2210d016 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:29.367164 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a729e-993c-463f-a884-23cfba801999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:29.367765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a729e-993c-463f-a884-23cfba801999 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:29.368034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a729e-993c-463f-a884-23cfba801999 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:29.368302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a729e-993c-463f-a884-23cfba801999 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:29.368976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a729e-993c-463f-a884-23cfba801999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:29.369273 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a729e-993c-463f-a884-23cfba801999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:29.370648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a729e-993c-463f-a884-23cfba801999 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:29.382416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 634/1270] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:29.391852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951d3b8-ff72-4f17-ba62-7c28f14b7a5f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:29.427104 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:29.519242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951d3b8-ff72-4f17-ba62-7c28f14b7a5f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:29.558290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd8e40-7103-43f8-9ee6-a2e376b5cccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:29.558940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd8e40-7103-43f8-9ee6-a2e376b5cccc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:29.559187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd8e40-7103-43f8-9ee6-a2e376b5cccc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:29.559456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd8e40-7103-43f8-9ee6-a2e376b5cccc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:29.566762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd8e40-7103-43f8-9ee6-a2e376b5cccc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:29.568234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd8e40-7103-43f8-9ee6-a2e376b5cccc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-e0fd8e40-7103-43f8-9ee6-a2e376b5cccc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:08:29.579848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:29.580921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:29.582466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:08:29.582466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:08:29.582466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:08:29.582466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:08:29.582466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:08:29.582466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:08:29.582466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:08:29.582466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:29.582466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 637/1271] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:08:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:29.591019 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a729e-993c-463f-a884-23cfba801999 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:29.651552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:29.698384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a729e-993c-463f-a884-23cfba801999 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:29.733835 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36ed6c-7300-4be7-9ae3-766ebc62d506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:29.734483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36ed6c-7300-4be7-9ae3-766ebc62d506 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:29.734781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36ed6c-7300-4be7-9ae3-766ebc62d506 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:29.735052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36ed6c-7300-4be7-9ae3-766ebc62d506 None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:29.735740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36ed6c-7300-4be7-9ae3-766ebc62d506 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:29.736050 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36ed6c-7300-4be7-9ae3-766ebc62d506 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:29.737845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36ed6c-7300-4be7-9ae3-766ebc62d506 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:29.749905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 635/1272] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:08:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:29.756301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd8e40-7103-43f8-9ee6-a2e376b5cccc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:29.860253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:29.874680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd8e40-7103-43f8-9ee6-a2e376b5cccc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:29.909205 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687afe0-aaae-4200-a1e7-ed94cfa4bccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:29.909781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687afe0-aaae-4200-a1e7-ed94cfa4bccf None admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:29.910052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687afe0-aaae-4200-a1e7-ed94cfa4bccf None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:29.910323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687afe0-aaae-4200-a1e7-ed94cfa4bccf None admin] PATH_INFO: `/v3/projects/27ac51280046479e82faaf3c0b550f72/users/2b8be1ec4a7a45b88d84bd9c48fab407/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:29.912477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687afe0-aaae-4200-a1e7-ed94cfa4bccf None admin] RBAC: Authorizing `identity:create_grant(project_id=27ac51280046479e82faaf3c0b550f72, user_id=2b8be1ec4a7a45b88d84bd9c48fab407, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:29.913979 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687afe0-aaae-4200-a1e7-ed94cfa4bccf None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:29.929764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 638/1273] 10.222.0.75 () {58 vars in 1439 bytes} [Fri Sep 23 15:08:29 2022] PUT /identity/v3/projects/27ac51280046479e82faaf3c0b550f72/users/2b8be1ec4a7a45b88d84bd9c48fab407/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:08:32.083547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac1dd5-e190-45fe-8e2d-919ec97c68bc None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:32.084451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac1dd5-e190-45fe-8e2d-919ec97c68bc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:32.084451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac1dd5-e190-45fe-8e2d-919ec97c68bc None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:32.086563 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 636/1274] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:08:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:08:32.094778 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6ba23-2246-4bbd-ac89-8744d76af924 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:32.095178 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6ba23-2246-4bbd-ac89-8744d76af924 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:32.095383 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6ba23-2246-4bbd-ac89-8744d76af924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:32.116011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f8f6ba23-2246-4bbd-ac89-8744d76af924 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:08:32.117222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f8f6ba23-2246-4bbd-ac89-8744d76af924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:08:32.221054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 639/1275] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:08:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:08:32.487802 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac1dd5-e190-45fe-8e2d-919ec97c68bc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:32.603820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac1dd5-e190-45fe-8e2d-919ec97c68bc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:32.622103 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:32.642227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87288f-0cf9-4c12-a303-626320673d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:32.642843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87288f-0cf9-4c12-a303-626320673d33 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:32.643112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87288f-0cf9-4c12-a303-626320673d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:32.643373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87288f-0cf9-4c12-a303-626320673d33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:32.653832 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87288f-0cf9-4c12-a303-626320673d33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-1a87288f-0cf9-4c12-a303-626320673d33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 15:08:32.658738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:32.659843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:32.660718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:32.660718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:32.660718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:32.660718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:32.660718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 15:08:32.660718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 15:08:32.660718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 15:08:32.660718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 15:08:32.660718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 15:08:32.660718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 15:08:32.660718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 15:08:32.660718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:32.660718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 637/1276] 10.222.0.75 () {56 vars in 1096 bytes} [Fri Sep 23 15:08:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:32.666271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6ba23-2246-4bbd-ac89-8744d76af924 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:32.752464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:32.771889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6ba23-2246-4bbd-ac89-8744d76af924 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:32.807614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f89958-e393-4d8e-97bf-0d37ed51d9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:32.808214 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f89958-e393-4d8e-97bf-0d37ed51d9ec None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:32.808479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f89958-e393-4d8e-97bf-0d37ed51d9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:32.808776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f89958-e393-4d8e-97bf-0d37ed51d9ec None admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:32.809432 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f89958-e393-4d8e-97bf-0d37ed51d9ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:32.809727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f89958-e393-4d8e-97bf-0d37ed51d9ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:32.811111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f89958-e393-4d8e-97bf-0d37ed51d9ec None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:32.819705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 640/1277] 10.222.0.75 () {56 vars in 1099 bytes} [Fri Sep 23 15:08:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:32.825951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87288f-0cf9-4c12-a303-626320673d33 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:32.935654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87288f-0cf9-4c12-a303-626320673d33 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:32.970928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bb533-7e7f-4112-badb-a91e4536f022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:32.971532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bb533-7e7f-4112-badb-a91e4536f022 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:32.971800 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bb533-7e7f-4112-badb-a91e4536f022 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:32.972067 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bb533-7e7f-4112-badb-a91e4536f022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:32.982506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bb533-7e7f-4112-badb-a91e4536f022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:32.983938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bb533-7e7f-4112-badb-a91e4536f022 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-389bb533-7e7f-4112-badb-a91e4536f022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:08:32.999153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:33.000257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:08:33.001374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:08:33.001374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:33.001374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:33.001374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 638/1278] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:33.008419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f89958-e393-4d8e-97bf-0d37ed51d9ec None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:33.114283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f89958-e393-4d8e-97bf-0d37ed51d9ec None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:33.150579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8aab0-0e23-403f-ab85-790d95068dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:33.151074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8aab0-0e23-403f-ab85-790d95068dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:33.151464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8aab0-0e23-403f-ab85-790d95068dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:33.151780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8aab0-0e23-403f-ab85-790d95068dd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:33.152701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8aab0-0e23-403f-ab85-790d95068dd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:33.153164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8aab0-0e23-403f-ab85-790d95068dd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:33.154983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8aab0-0e23-403f-ab85-790d95068dd9 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:33.172232 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 641/1279] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:33.179109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bb533-7e7f-4112-badb-a91e4536f022 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:33.211890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:33.285402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bb533-7e7f-4112-badb-a91e4536f022 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:33.324182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767c334-8d46-4b0d-9a81-f8811c378605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:33.324828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767c334-8d46-4b0d-9a81-f8811c378605 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:33.325101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767c334-8d46-4b0d-9a81-f8811c378605 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:33.325364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767c334-8d46-4b0d-9a81-f8811c378605 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:33.332889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767c334-8d46-4b0d-9a81-f8811c378605 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:33.334339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767c334-8d46-4b0d-9a81-f8811c378605 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-7767c334-8d46-4b0d-9a81-f8811c378605 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 15:08:33.346154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:33.347222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:33.348249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:33.348249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:33.348249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 15:08:33.348249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 15:08:33.348249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 15:08:33.348249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 15:08:33.348249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 15:08:33.348249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 15:08:33.348249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 15:08:33.348249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:33.348249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 639/1280] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:33.353692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8aab0-0e23-403f-ab85-790d95068dd9 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:33.358295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:33.460931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8aab0-0e23-403f-ab85-790d95068dd9 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:33.498430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e58484-c28c-47dd-9bec-acb9d79122e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:33.499012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58484-c28c-47dd-9bec-acb9d79122e8 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:33.499281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58484-c28c-47dd-9bec-acb9d79122e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:33.499542 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58484-c28c-47dd-9bec-acb9d79122e8 None admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:33.500979 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e58484-c28c-47dd-9bec-acb9d79122e8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:33.501276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e58484-c28c-47dd-9bec-acb9d79122e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:33.502692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e58484-c28c-47dd-9bec-acb9d79122e8 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:33.520073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 642/1281] 10.222.0.75 () {56 vars in 1143 bytes} [Fri Sep 23 15:08:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:33.529629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767c334-8d46-4b0d-9a81-f8811c378605 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:33.631412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:33.642962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767c334-8d46-4b0d-9a81-f8811c378605 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:33.676192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122207f-2f5b-4639-83a5-4a32737a9e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:33.676830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122207f-2f5b-4639-83a5-4a32737a9e75 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:33.677104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122207f-2f5b-4639-83a5-4a32737a9e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:33.677370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122207f-2f5b-4639-83a5-4a32737a9e75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:33.750916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122207f-2f5b-4639-83a5-4a32737a9e75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:33.752361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122207f-2f5b-4639-83a5-4a32737a9e75 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:33.842744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:33.854762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 640/1282] 10.222.0.75 () {58 vars in 1282 bytes} [Fri Sep 23 15:08:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 15:08:33.863161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e58484-c28c-47dd-9bec-acb9d79122e8 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:33.926183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:33.964534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e58484-c28c-47dd-9bec-acb9d79122e8 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:34.000239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe83ea-cba7-4faf-a59d-0839742fb22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:34.000918 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe83ea-cba7-4faf-a59d-0839742fb22f None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:34.001185 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe83ea-cba7-4faf-a59d-0839742fb22f None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:34.001448 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe83ea-cba7-4faf-a59d-0839742fb22f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:34.012146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe83ea-cba7-4faf-a59d-0839742fb22f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:34.013607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe83ea-cba7-4faf-a59d-0839742fb22f None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-8bfe83ea-cba7-4faf-a59d-0839742fb22f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 15:08:34.028382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:34.029456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 15:08:34.030574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 15:08:34.030574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 15:08:34.030574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:34.030574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 643/1283] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 15:08:34.036364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122207f-2f5b-4639-83a5-4a32737a9e75 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:34.143887 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122207f-2f5b-4639-83a5-4a32737a9e75 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:34.180971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c70b51-406e-4eff-8a27-eb62ebbf2c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:34.181629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c70b51-406e-4eff-8a27-eb62ebbf2c66 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:34.181820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c70b51-406e-4eff-8a27-eb62ebbf2c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:34.182085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c70b51-406e-4eff-8a27-eb62ebbf2c66 None admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:34.182724 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c70b51-406e-4eff-8a27-eb62ebbf2c66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:34.183017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c70b51-406e-4eff-8a27-eb62ebbf2c66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:34.184382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c70b51-406e-4eff-8a27-eb62ebbf2c66 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:34.195732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 641/1284] 10.222.0.75 () {56 vars in 1109 bytes} [Fri Sep 23 15:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:34.202454 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe83ea-cba7-4faf-a59d-0839742fb22f None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:34.311665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe83ea-cba7-4faf-a59d-0839742fb22f None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:34.325639 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:34.354036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e37fc5-3f3f-420f-82fe-04d736a34483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:34.354720 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e37fc5-3f3f-420f-82fe-04d736a34483 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:34.354993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e37fc5-3f3f-420f-82fe-04d736a34483 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:34.355246 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e37fc5-3f3f-420f-82fe-04d736a34483 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:34.362478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e37fc5-3f3f-420f-82fe-04d736a34483 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:34.363892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e37fc5-3f3f-420f-82fe-04d736a34483 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application [None req-65e37fc5-3f3f-420f-82fe-04d736a34483 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 15:08:34.375116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:34.376119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:34.377364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 15:08:34.377364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 15:08:34.377364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 15:08:34.377364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 15:08:34.377364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 15:08:34.377364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 15:08:34.377364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 15:08:34.377364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: ERROR keystone.server.flask.application Sep 23 15:08:34.377364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 644/1285] 10.222.0.75 () {56 vars in 1111 bytes} [Fri Sep 23 15:08:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:34.384064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c70b51-406e-4eff-8a27-eb62ebbf2c66 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:34.436994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:34.487739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c70b51-406e-4eff-8a27-eb62ebbf2c66 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:34.523877 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51d62c-4e93-4f5a-9625-8efd1b02cefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:34.524467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51d62c-4e93-4f5a-9625-8efd1b02cefb None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:34.524764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51d62c-4e93-4f5a-9625-8efd1b02cefb None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:34.525028 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51d62c-4e93-4f5a-9625-8efd1b02cefb None admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:34.525714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51d62c-4e93-4f5a-9625-8efd1b02cefb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:34.526020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51d62c-4e93-4f5a-9625-8efd1b02cefb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:34.527441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51d62c-4e93-4f5a-9625-8efd1b02cefb None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:34.539777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 642/1286] 10.222.0.75 () {56 vars in 1148 bytes} [Fri Sep 23 15:08:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:34.546133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e37fc5-3f3f-420f-82fe-04d736a34483 None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:34.660220 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e37fc5-3f3f-420f-82fe-04d736a34483 None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:34.699574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96615361-21af-4b8b-a332-84e1621587b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:34.700101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96615361-21af-4b8b-a332-84e1621587b6 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:34.700368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96615361-21af-4b8b-a332-84e1621587b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:34.700636 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96615361-21af-4b8b-a332-84e1621587b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:34.701393 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96615361-21af-4b8b-a332-84e1621587b6 None admin] RBAC: Adding query filter params (role.id=ae444a489623434999d758b71ae2dc87, scope.project.id=27ac51280046479e82faaf3c0b550f72, user.id=2b8be1ec4a7a45b88d84bd9c48fab407) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:34.701700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96615361-21af-4b8b-a332-84e1621587b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:34.703190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96615361-21af-4b8b-a332-84e1621587b6 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:34.711129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 645/1287] 10.222.0.75 () {56 vars in 1374 bytes} [Fri Sep 23 15:08:34 2022] GET /identity/v3/role_assignments?user.id=2b8be1ec4a7a45b88d84bd9c48fab407&scope.project.id=27ac51280046479e82faaf3c0b550f72&role.id=ae444a489623434999d758b71ae2dc87 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:36.876897 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfab3fe-25f9-4e6b-83c8-dc96e84f0516 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:36.877741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfab3fe-25f9-4e6b-83c8-dc96e84f0516 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:36.877741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfab3fe-25f9-4e6b-83c8-dc96e84f0516 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:36.879845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 643/1288] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:08:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:08:36.886168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a6bb0-6210-4e4c-9a2f-b0cee6aba2b4 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:36.886934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a6bb0-6210-4e4c-9a2f-b0cee6aba2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:36.887096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a6bb0-6210-4e4c-9a2f-b0cee6aba2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:36.908084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-065a6bb0-6210-4e4c-9a2f-b0cee6aba2b4 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:08:36.909299 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-065a6bb0-6210-4e4c-9a2f-b0cee6aba2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:08:36.941529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:36.985094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 646/1289] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:08:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:08:37.260726 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfab3fe-25f9-4e6b-83c8-dc96e84f0516 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:37.365830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfab3fe-25f9-4e6b-83c8-dc96e84f0516 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:37.376940 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:37.401483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b6eef-1a21-4a47-b099-487b2b2fb0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:37.402123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b6eef-1a21-4a47-b099-487b2b2fb0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:37.402390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b6eef-1a21-4a47-b099-487b2b2fb0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:37.402650 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b6eef-1a21-4a47-b099-487b2b2fb0b1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:37.403232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b6eef-1a21-4a47-b099-487b2b2fb0b1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:37.404624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b6eef-1a21-4a47-b099-487b2b2fb0b1 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-3a6b6eef-1a21-4a47-b099-487b2b2fb0b1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:37.413574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:37.414764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:37.415873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 644/1290] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Sep 23 15:08:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:37.421172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a6bb0-6210-4e4c-9a2f-b0cee6aba2b4 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:37.533946 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a6bb0-6210-4e4c-9a2f-b0cee6aba2b4 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:37.570601 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd3b3b-3d64-41cc-bdfe-6c86d3cdda9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:37.571208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd3b3b-3d64-41cc-bdfe-6c86d3cdda9c None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:37.571477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd3b3b-3d64-41cc-bdfe-6c86d3cdda9c None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:37.571737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd3b3b-3d64-41cc-bdfe-6c86d3cdda9c None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:37.572364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd3b3b-3d64-41cc-bdfe-6c86d3cdda9c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:37.572711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd3b3b-3d64-41cc-bdfe-6c86d3cdda9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:37.574061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd3b3b-3d64-41cc-bdfe-6c86d3cdda9c None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:37.578940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 647/1291] 10.222.0.75 () {56 vars in 1116 bytes} [Fri Sep 23 15:08:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:37.585390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b6eef-1a21-4a47-b099-487b2b2fb0b1 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:37.702245 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b6eef-1a21-4a47-b099-487b2b2fb0b1 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:37.739338 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835be592-b11e-47ca-9031-a53bbc703434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:37.739925 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835be592-b11e-47ca-9031-a53bbc703434 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:37.740190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835be592-b11e-47ca-9031-a53bbc703434 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:37.740451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835be592-b11e-47ca-9031-a53bbc703434 None admin] PATH_INFO: `/v3/services` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:37.741109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835be592-b11e-47ca-9031-a53bbc703434 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:37.741406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835be592-b11e-47ca-9031-a53bbc703434 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:37.742762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835be592-b11e-47ca-9031-a53bbc703434 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:37.747255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 645/1292] 10.222.0.75 () {56 vars in 1116 bytes} [Fri Sep 23 15:08:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:39.883264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c564c7a-2648-419e-908a-054f90ddf3cf None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:39.883815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c564c7a-2648-419e-908a-054f90ddf3cf None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:39.883815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c564c7a-2648-419e-908a-054f90ddf3cf None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:39.885191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 648/1293] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:08:39.890785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dc00a-beb5-47e4-a142-27a4589e95f6 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:39.891239 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dc00a-beb5-47e4-a142-27a4589e95f6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:39.891555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dc00a-beb5-47e4-a142-27a4589e95f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:39.911894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2d9dc00a-beb5-47e4-a142-27a4589e95f6 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:08:39.913099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2d9dc00a-beb5-47e4-a142-27a4589e95f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:08:39.949477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:39.992577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 646/1294] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:08:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:08:40.272792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c564c7a-2648-419e-908a-054f90ddf3cf None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:40.307461 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:40.385473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c564c7a-2648-419e-908a-054f90ddf3cf None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:40.420910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858d99e-1b90-4ad5-a225-50006cc711bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:40.421475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858d99e-1b90-4ad5-a225-50006cc711bc None admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:40.422123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858d99e-1b90-4ad5-a225-50006cc711bc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:40.422395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858d99e-1b90-4ad5-a225-50006cc711bc None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:40.423047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858d99e-1b90-4ad5-a225-50006cc711bc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:40.424414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858d99e-1b90-4ad5-a225-50006cc711bc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:40.444481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 649/1295] 10.222.0.75 () {60 vars in 1141 bytes} [Fri Sep 23 15:08:40 2022] POST /identity/v3/services => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:08:42.588044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c3809-fff3-4f6d-82aa-56514cacfaa4 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:42.588610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c3809-fff3-4f6d-82aa-56514cacfaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:42.588610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c3809-fff3-4f6d-82aa-56514cacfaa4 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:42.590027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 647/1296] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:08:42.596433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dfcfc-f7ff-476e-b62d-8ec76603ce32 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:42.597207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dfcfc-f7ff-476e-b62d-8ec76603ce32 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:42.597321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2dfcfc-f7ff-476e-b62d-8ec76603ce32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:42.627185 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-dc2dfcfc-f7ff-476e-b62d-8ec76603ce32 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:08:42.628371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dc2dfcfc-f7ff-476e-b62d-8ec76603ce32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:08:42.708154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 650/1297] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:08:42 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:08:42.977112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c3809-fff3-4f6d-82aa-56514cacfaa4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:43.098206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c3809-fff3-4f6d-82aa-56514cacfaa4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:43.133891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedf74c-5e4d-4f37-98d3-40d94575dc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:43.134506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedf74c-5e4d-4f37-98d3-40d94575dc58 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:43.134778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedf74c-5e4d-4f37-98d3-40d94575dc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:43.135043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedf74c-5e4d-4f37-98d3-40d94575dc58 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:43.135623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedf74c-5e4d-4f37-98d3-40d94575dc58 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:43.137092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cedf74c-5e4d-4f37-98d3-40d94575dc58 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-2cedf74c-5e4d-4f37-98d3-40d94575dc58 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:43.146150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:43.148018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:43.149643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 648/1298] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Sep 23 15:08:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:43.153648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dfcfc-f7ff-476e-b62d-8ec76603ce32 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:43.174190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:43.285496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2dfcfc-f7ff-476e-b62d-8ec76603ce32 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:43.326828 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15558e20-e59e-4363-9f11-bfe27a70e8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:43.327428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15558e20-e59e-4363-9f11-bfe27a70e8dc None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:43.327700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15558e20-e59e-4363-9f11-bfe27a70e8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:43.328007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15558e20-e59e-4363-9f11-bfe27a70e8dc None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:43.328666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15558e20-e59e-4363-9f11-bfe27a70e8dc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:43.328967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15558e20-e59e-4363-9f11-bfe27a70e8dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:43.330325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15558e20-e59e-4363-9f11-bfe27a70e8dc None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:43.335953 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 651/1299] 10.222.0.75 () {56 vars in 1116 bytes} [Fri Sep 23 15:08:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:43.343165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedf74c-5e4d-4f37-98d3-40d94575dc58 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:43.376102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:43.459687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedf74c-5e4d-4f37-98d3-40d94575dc58 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:43.495026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a30c1-7e13-42c9-8a05-1dcaad1b219b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:43.495614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a30c1-7e13-42c9-8a05-1dcaad1b219b None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:43.495885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a30c1-7e13-42c9-8a05-1dcaad1b219b None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:43.496151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a30c1-7e13-42c9-8a05-1dcaad1b219b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:43.496888 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a30c1-7e13-42c9-8a05-1dcaad1b219b None admin] RBAC: Adding query filter params (interface=public, service_id=8cfd38049d7c413cb9d96a6365c80acd, region_id=RegionOne) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:43.497191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a30c1-7e13-42c9-8a05-1dcaad1b219b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:43.498553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a30c1-7e13-42c9-8a05-1dcaad1b219b None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:43.504209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 649/1300] 10.222.0.75 () {56 vars in 1251 bytes} [Fri Sep 23 15:08:43 2022] GET /identity/v3/endpoints?service_id=8cfd38049d7c413cb9d96a6365c80acd&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:43.513580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15558e20-e59e-4363-9f11-bfe27a70e8dc None admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:43.607799 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:43.627387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15558e20-e59e-4363-9f11-bfe27a70e8dc None admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:43.662096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f043c-7658-4418-bf5a-9d8968434488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:43.662693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f043c-7658-4418-bf5a-9d8968434488 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:43.662972 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f043c-7658-4418-bf5a-9d8968434488 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:43.664248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f043c-7658-4418-bf5a-9d8968434488 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:43.664887 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f043c-7658-4418-bf5a-9d8968434488 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:43.666239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f043c-7658-4418-bf5a-9d8968434488 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:43.673833 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 652/1301] 10.222.0.75 () {56 vars in 1087 bytes} [Fri Sep 23 15:08:43 2022] GET /identity/v3/services => generated 2191 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:08:45.785237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c26524-cad8-47cb-99f6-0d03fcc70176 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:45.786192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c26524-cad8-47cb-99f6-0d03fcc70176 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:45.786192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c26524-cad8-47cb-99f6-0d03fcc70176 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:45.788193 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 650/1302] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:08:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:08:45.794542 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed611037-a83f-488b-8a88-ef311ba5456f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:45.795004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed611037-a83f-488b-8a88-ef311ba5456f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:45.795108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed611037-a83f-488b-8a88-ef311ba5456f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:45.821637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ed611037-a83f-488b-8a88-ef311ba5456f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:08:45.822827 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ed611037-a83f-488b-8a88-ef311ba5456f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:08:45.892173 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 653/1303] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:08:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 15:08:46.187131 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c26524-cad8-47cb-99f6-0d03fcc70176 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:46.291679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:46.299397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c26524-cad8-47cb-99f6-0d03fcc70176 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:46.335120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be011e-b697-4f69-9ea6-a4ceb7963fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:46.335736 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be011e-b697-4f69-9ea6-a4ceb7963fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:46.336007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be011e-b697-4f69-9ea6-a4ceb7963fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:46.336275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be011e-b697-4f69-9ea6-a4ceb7963fb0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:46.336953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be011e-b697-4f69-9ea6-a4ceb7963fb0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:46.338314 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be011e-b697-4f69-9ea6-a4ceb7963fb0 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application [None req-e2be011e-b697-4f69-9ea6-a4ceb7963fb0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 15:08:46.347238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 15:08:46.348425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application with Lock( Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self._enter() Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application return self.creator() Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application created_value = creator( Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 15:08:46.349790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: ERROR keystone.server.flask.application Sep 23 15:08:46.352773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 651/1304] 10.222.0.75 () {56 vars in 1117 bytes} [Fri Sep 23 15:08:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 15:08:46.354885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed611037-a83f-488b-8a88-ef311ba5456f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:46.442411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:46.469141 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed611037-a83f-488b-8a88-ef311ba5456f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:46.511826 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7713b3-d201-4308-860f-16182ba7e575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:46.512424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7713b3-d201-4308-860f-16182ba7e575 None admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:46.512731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7713b3-d201-4308-860f-16182ba7e575 None admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:46.513000 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7713b3-d201-4308-860f-16182ba7e575 None admin] PATH_INFO: `/v3/services` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:46.513636 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7713b3-d201-4308-860f-16182ba7e575 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:08:46.513936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7713b3-d201-4308-860f-16182ba7e575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:46.515280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7713b3-d201-4308-860f-16182ba7e575 None admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:46.520916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 654/1305] 10.222.0.75 () {56 vars in 1116 bytes} [Fri Sep 23 15:08:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:08:46.527150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be011e-b697-4f69-9ea6-a4ceb7963fb0 None admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:08:46.634736 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be011e-b697-4f69-9ea6-a4ceb7963fb0 None admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:08:46.669398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe74366-70ee-4395-beef-241da1d218b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:08:46.669990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe74366-70ee-4395-beef-241da1d218b5 None admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:08:46.670255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe74366-70ee-4395-beef-241da1d218b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:08:46.670515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe74366-70ee-4395-beef-241da1d218b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:08:46.671177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe74366-70ee-4395-beef-241da1d218b5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:08:46.672787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe74366-70ee-4395-beef-241da1d218b5 None admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:08:46.700340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 652/1306] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:08:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:08:56.517440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=30 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:08:56.692570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:09:51.627952 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df801b7-bf12-418c-8850-a2b65e48f1a5 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:09:51.629031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df801b7-bf12-418c-8850-a2b65e48f1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:09:51.629607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df801b7-bf12-418c-8850-a2b65e48f1a5 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:09:51.632393 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 655/1307] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:09:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:09:51.637926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130d964-2f70-4405-8f05-2910de3f857f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:09:51.638430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130d964-2f70-4405-8f05-2910de3f857f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:09:51.638526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130d964-2f70-4405-8f05-2910de3f857f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:09:51.801253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3130d964-2f70-4405-8f05-2910de3f857f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:09:51.804186 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3130d964-2f70-4405-8f05-2910de3f857f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:09:51.923157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 653/1308] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:09:52.218028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fc081-b2dd-4e4c-aaa9-38dbc6e9a37f None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:09:52.218689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fc081-b2dd-4e4c-aaa9-38dbc6e9a37f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:09:52.219155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fc081-b2dd-4e4c-aaa9-38dbc6e9a37f None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:09:52.220960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 656/1309] 10.222.0.75 () {54 vars in 964 bytes} [Fri Sep 23 15:09:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:09:52.227417 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dce419-6929-4432-b59d-79c3a0bfba7a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:09:52.228011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dce419-6929-4432-b59d-79c3a0bfba7a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:09:52.228505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dce419-6929-4432-b59d-79c3a0bfba7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:09:52.313313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c9dce419-6929-4432-b59d-79c3a0bfba7a None None] MFA Rules not processed for user `02afdedf8d394bbfa977ab7a22fafea7`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:09:52.314871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c9dce419-6929-4432-b59d-79c3a0bfba7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:09:52.337375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:09:52.398273 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 654/1310] 10.222.0.75 () {58 vars in 1061 bytes} [Fri Sep 23 15:09:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:09:52.407588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fc081-b2dd-4e4c-aaa9-38dbc6e9a37f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:09:52.409985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-848fc081-b2dd-4e4c-aaa9-38dbc6e9a37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:09:52.539866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fc081-b2dd-4e4c-aaa9-38dbc6e9a37f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:09:52.628417 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847cf6e-c6c6-430d-9e87-0fbcd0c3aaf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:09:52.629134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847cf6e-c6c6-430d-9e87-0fbcd0c3aaf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:09:52.629327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847cf6e-c6c6-430d-9e87-0fbcd0c3aaf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:09:52.629840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847cf6e-c6c6-430d-9e87-0fbcd0c3aaf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:09:52.632047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4847cf6e-c6c6-430d-9e87-0fbcd0c3aaf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:09:52.692569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:09:52.751726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847cf6e-c6c6-430d-9e87-0fbcd0c3aaf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:09:52.753479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847cf6e-c6c6-430d-9e87-0fbcd0c3aaf7 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:09:52.775484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 657/1311] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:09:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:09:55.233595 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf68ffe-986d-44b4-af52-82b7f52cdf01 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:09:55.234148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf68ffe-986d-44b4-af52-82b7f52cdf01 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:09:55.234148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf68ffe-986d-44b4-af52-82b7f52cdf01 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:09:55.235379 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 655/1312] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:09:55.243068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a40f0-918b-464c-93fd-05d673151ea0 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:09:55.243500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a40f0-918b-464c-93fd-05d673151ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:09:55.243615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a40f0-918b-464c-93fd-05d673151ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:09:55.342700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b40a40f0-918b-464c-93fd-05d673151ea0 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:09:55.343895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b40a40f0-918b-464c-93fd-05d673151ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:09:55.449764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 658/1313] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:09:55.732965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf68ffe-986d-44b4-af52-82b7f52cdf01 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:09:55.734571 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5bf68ffe-986d-44b4-af52-82b7f52cdf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:09:55.841940 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf68ffe-986d-44b4-af52-82b7f52cdf01 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:09:55.919503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:09:55.923757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf574373-e8f1-4b9d-a7e1-144d5a71051c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:09:55.924692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf574373-e8f1-4b9d-a7e1-144d5a71051c service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:09:55.924977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf574373-e8f1-4b9d-a7e1-144d5a71051c service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:09:55.925251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf574373-e8f1-4b9d-a7e1-144d5a71051c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:09:55.927607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bf574373-e8f1-4b9d-a7e1-144d5a71051c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:09:56.040083 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf574373-e8f1-4b9d-a7e1-144d5a71051c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:09:56.041571 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf574373-e8f1-4b9d-a7e1-144d5a71051c service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:09:56.061026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 656/1314] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:09:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:09:58.790478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b4ea4-5215-462a-8b61-ac7b3e915b64 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:09:58.791379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b4ea4-5215-462a-8b61-ac7b3e915b64 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:09:58.791379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b4ea4-5215-462a-8b61-ac7b3e915b64 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:09:58.793587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 659/1315] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:09:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:09:58.802568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d45139-3a9f-4e8a-9f76-87d4192716b1 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:09:58.803002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d45139-3a9f-4e8a-9f76-87d4192716b1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:09:58.803093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d45139-3a9f-4e8a-9f76-87d4192716b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:09:58.878298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-37d45139-3a9f-4e8a-9f76-87d4192716b1 None None] MFA Rules not processed for user `4cdec6738f534b16af19e42f8f462d49`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:09:58.879495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-37d45139-3a9f-4e8a-9f76-87d4192716b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:09:58.938454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:09:58.996787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 657/1316] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:09:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:09:59.283358 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b4ea4-5215-462a-8b61-ac7b3e915b64 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:09:59.285932 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fc4b4ea4-5215-462a-8b61-ac7b3e915b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:09:59.296828 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:09:59.414159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b4ea4-5215-462a-8b61-ac7b3e915b64 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:09:59.498840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7be9f-3774-4025-b1fc-63e49c036645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:09:59.499443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7be9f-3774-4025-b1fc-63e49c036645 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:09:59.499721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7be9f-3774-4025-b1fc-63e49c036645 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:09:59.499989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7be9f-3774-4025-b1fc-63e49c036645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:09:59.501617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c7e7be9f-3774-4025-b1fc-63e49c036645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:09:59.569412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:09:59.617568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7be9f-3774-4025-b1fc-63e49c036645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:09:59.619048 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7be9f-3774-4025-b1fc-63e49c036645 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:09:59.637961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 660/1317] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:09:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:10:02.342395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf048ba-21e7-41aa-950e-56bbba8f2d1d None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:02.342977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf048ba-21e7-41aa-950e-56bbba8f2d1d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:02.342977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf048ba-21e7-41aa-950e-56bbba8f2d1d None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:02.344246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 658/1318] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:10:02.350864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306ae2d-4d51-47de-9777-68c73be8788a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:02.351465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306ae2d-4d51-47de-9777-68c73be8788a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:02.352075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306ae2d-4d51-47de-9777-68c73be8788a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:02.436912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b306ae2d-4d51-47de-9777-68c73be8788a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:02.438881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b306ae2d-4d51-47de-9777-68c73be8788a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:02.549712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 661/1319] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:02.833407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf048ba-21e7-41aa-950e-56bbba8f2d1d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:10:02.835507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bdf048ba-21e7-41aa-950e-56bbba8f2d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:02.951733 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf048ba-21e7-41aa-950e-56bbba8f2d1d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:10:03.030579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b02952-bb1c-4340-b703-78b0ebf663c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:10:03.031181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b02952-bb1c-4340-b703-78b0ebf663c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:03.031454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b02952-bb1c-4340-b703-78b0ebf663c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:03.031723 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b02952-bb1c-4340-b703-78b0ebf663c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:03.033398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c5b02952-bb1c-4340-b703-78b0ebf663c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:03.040425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:03.149825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b02952-bb1c-4340-b703-78b0ebf663c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:10:03.151191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b02952-bb1c-4340-b703-78b0ebf663c2 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:10:03.170378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 659/1320] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:10:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:10:06.006363 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3def744-6fd3-421d-bd4f-9ac0346d5133 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:06.006909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3def744-6fd3-421d-bd4f-9ac0346d5133 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:06.007186 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3def744-6fd3-421d-bd4f-9ac0346d5133 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:06.008564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 662/1321] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:06.014018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c5c2f-5ed7-4a57-8c9e-c7a5b1d2bc58 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:06.014514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c5c2f-5ed7-4a57-8c9e-c7a5b1d2bc58 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:06.014514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c5c2f-5ed7-4a57-8c9e-c7a5b1d2bc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:06.094449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a78c5c2f-5ed7-4a57-8c9e-c7a5b1d2bc58 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:06.095705 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a78c5c2f-5ed7-4a57-8c9e-c7a5b1d2bc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:06.164181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:06.213443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 660/1322] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:10:06.509227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb385db6-7e09-4729-87cb-d8da517f82c9 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:06.509526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb385db6-7e09-4729-87cb-d8da517f82c9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:06.509790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb385db6-7e09-4729-87cb-d8da517f82c9 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:06.511190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 663/1323] 10.222.0.75 () {54 vars in 964 bytes} [Fri Sep 23 15:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:06.517314 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168b9cc-54e2-4ebc-ac79-0b3516ee53d7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:06.517588 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168b9cc-54e2-4ebc-ac79-0b3516ee53d7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:06.517857 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168b9cc-54e2-4ebc-ac79-0b3516ee53d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:06.600926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9168b9cc-54e2-4ebc-ac79-0b3516ee53d7 None None] MFA Rules not processed for user `02afdedf8d394bbfa977ab7a22fafea7`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:06.602038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9168b9cc-54e2-4ebc-ac79-0b3516ee53d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:06.680941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 661/1324] 10.222.0.75 () {58 vars in 1061 bytes} [Fri Sep 23 15:10:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:06.690384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb385db6-7e09-4729-87cb-d8da517f82c9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:10:06.691965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cb385db6-7e09-4729-87cb-d8da517f82c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:06.713319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:06.802490 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb385db6-7e09-4729-87cb-d8da517f82c9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:10:06.877095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75ca77-63a7-40c8-9f99-f3268b9afff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:10:06.877629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75ca77-63a7-40c8-9f99-f3268b9afff5 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:06.877893 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75ca77-63a7-40c8-9f99-f3268b9afff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:06.878127 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75ca77-63a7-40c8-9f99-f3268b9afff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:06.879741 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8b75ca77-63a7-40c8-9f99-f3268b9afff5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:06.953004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:06.994786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75ca77-63a7-40c8-9f99-f3268b9afff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:10:06.996224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75ca77-63a7-40c8-9f99-f3268b9afff5 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:10:07.014435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 664/1325] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:10:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:10:09.416312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c33e0-f128-4197-b3e5-9a913ee58a44 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:09.416909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c33e0-f128-4197-b3e5-9a913ee58a44 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:09.416909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c33e0-f128-4197-b3e5-9a913ee58a44 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:09.418130 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 662/1326] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:09.423567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dea54b-a868-45a3-aa1b-e7c773b01c52 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:09.424015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dea54b-a868-45a3-aa1b-e7c773b01c52 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:09.424109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dea54b-a868-45a3-aa1b-e7c773b01c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:09.494537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b8dea54b-a868-45a3-aa1b-e7c773b01c52 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:09.495786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b8dea54b-a868-45a3-aa1b-e7c773b01c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:09.596738 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 665/1327] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:09.870063 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c33e0-f128-4197-b3e5-9a913ee58a44 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:10:09.871618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-590c33e0-f128-4197-b3e5-9a913ee58a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:09.934858 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:09.979624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c33e0-f128-4197-b3e5-9a913ee58a44 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:10:10.055330 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970598a-408c-4f44-bda2-de7e4a9bb0a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:10:10.055951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970598a-408c-4f44-bda2-de7e4a9bb0a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:10.056189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970598a-408c-4f44-bda2-de7e4a9bb0a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:10.056485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970598a-408c-4f44-bda2-de7e4a9bb0a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:10.058135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1970598a-408c-4f44-bda2-de7e4a9bb0a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:10.170888 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:10.173408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970598a-408c-4f44-bda2-de7e4a9bb0a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:10:10.175208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970598a-408c-4f44-bda2-de7e4a9bb0a0 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:10:10.194159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 663/1328] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:10:12.403715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b03ff-1ecf-4e8b-8998-ffedf74d1456 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:12.404283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b03ff-1ecf-4e8b-8998-ffedf74d1456 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:12.404283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b03ff-1ecf-4e8b-8998-ffedf74d1456 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:12.405896 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 666/1329] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:12.411439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fd36c-6a59-4315-a9c0-bc62b458e91d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:12.412004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fd36c-6a59-4315-a9c0-bc62b458e91d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:12.412004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fd36c-6a59-4315-a9c0-bc62b458e91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:12.482166 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-442fd36c-6a59-4315-a9c0-bc62b458e91d None None] MFA Rules not processed for user `4cdec6738f534b16af19e42f8f462d49`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:12.483375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-442fd36c-6a59-4315-a9c0-bc62b458e91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:12.585435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 664/1330] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:12.867580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b03ff-1ecf-4e8b-8998-ffedf74d1456 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:10:12.869215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-aa9b03ff-1ecf-4e8b-8998-ffedf74d1456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:12.901697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:12.981435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b03ff-1ecf-4e8b-8998-ffedf74d1456 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:10:13.059666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62317c2-c890-4d49-b6b9-0215874aa761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:10:13.060270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62317c2-c890-4d49-b6b9-0215874aa761 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:13.060534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62317c2-c890-4d49-b6b9-0215874aa761 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:13.060820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62317c2-c890-4d49-b6b9-0215874aa761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:13.062397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b62317c2-c890-4d49-b6b9-0215874aa761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:13.145815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:13.180951 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62317c2-c890-4d49-b6b9-0215874aa761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:10:13.182398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62317c2-c890-4d49-b6b9-0215874aa761 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:10:13.201078 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 667/1331] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:10:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:10:15.755038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ef8ff-545a-4f6d-bb16-a436ba5ae93b None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:15.755917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ef8ff-545a-4f6d-bb16-a436ba5ae93b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:15.755917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ef8ff-545a-4f6d-bb16-a436ba5ae93b None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:15.758090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 665/1332] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:15.764146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5aee7-4b20-4d8c-8ae2-099dc800a4aa None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:15.764570 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5aee7-4b20-4d8c-8ae2-099dc800a4aa None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:15.764723 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5aee7-4b20-4d8c-8ae2-099dc800a4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:15.835586 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-fec5aee7-4b20-4d8c-8ae2-099dc800a4aa None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:15.836802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fec5aee7-4b20-4d8c-8ae2-099dc800a4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:15.948964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 668/1333] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:16.229259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ef8ff-545a-4f6d-bb16-a436ba5ae93b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:10:16.231731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d07ef8ff-545a-4f6d-bb16-a436ba5ae93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:16.310325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:16.365692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ef8ff-545a-4f6d-bb16-a436ba5ae93b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:10:16.441921 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38411f3-cf10-47fc-b986-8a89aba686fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:10:16.442527 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38411f3-cf10-47fc-b986-8a89aba686fb service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:16.442796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38411f3-cf10-47fc-b986-8a89aba686fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:16.443064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38411f3-cf10-47fc-b986-8a89aba686fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:16.444826 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f38411f3-cf10-47fc-b986-8a89aba686fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:16.570449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38411f3-cf10-47fc-b986-8a89aba686fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:10:16.571910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38411f3-cf10-47fc-b986-8a89aba686fb service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:10:16.583567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:16.593379 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 666/1334] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:10:18.772774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee575f95-445b-4bc7-9f20-e4c38db551ad None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:18.773381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee575f95-445b-4bc7-9f20-e4c38db551ad None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:18.773381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee575f95-445b-4bc7-9f20-e4c38db551ad None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:18.775371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 669/1335] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:18.784117 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b7a7f-aaa5-497f-9e16-71074dde7b81 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:18.784516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b7a7f-aaa5-497f-9e16-71074dde7b81 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:18.784739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b7a7f-aaa5-497f-9e16-71074dde7b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:18.855971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-019b7a7f-aaa5-497f-9e16-71074dde7b81 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:18.857245 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-019b7a7f-aaa5-497f-9e16-71074dde7b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:18.962063 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 667/1336] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:19.260120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee575f95-445b-4bc7-9f20-e4c38db551ad None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:10:19.262630 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ee575f95-445b-4bc7-9f20-e4c38db551ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:19.309245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:19.380633 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee575f95-445b-4bc7-9f20-e4c38db551ad None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:10:19.454596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67b347-6a73-4cb6-94ec-56c5b2872df5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:10:19.455195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67b347-6a73-4cb6-94ec-56c5b2872df5 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:19.455466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67b347-6a73-4cb6-94ec-56c5b2872df5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:19.455730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67b347-6a73-4cb6-94ec-56c5b2872df5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:19.457355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fa67b347-6a73-4cb6-94ec-56c5b2872df5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:19.546449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:19.573314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67b347-6a73-4cb6-94ec-56c5b2872df5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:10:19.574776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa67b347-6a73-4cb6-94ec-56c5b2872df5 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:10:19.593230 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 670/1337] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:10:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:10:22.212868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f95d78-5e8f-4a29-a631-e43b02867154 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:22.213788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f95d78-5e8f-4a29-a631-e43b02867154 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:22.213788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f95d78-5e8f-4a29-a631-e43b02867154 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:22.215806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 668/1338] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:22.224394 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bade7be-fcc4-4f1f-91cd-04712fee2f61 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:22.225192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bade7be-fcc4-4f1f-91cd-04712fee2f61 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:22.225628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bade7be-fcc4-4f1f-91cd-04712fee2f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:22.298970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6bade7be-fcc4-4f1f-91cd-04712fee2f61 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:22.300175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6bade7be-fcc4-4f1f-91cd-04712fee2f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:22.400164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 671/1339] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:22.681012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f95d78-5e8f-4a29-a631-e43b02867154 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:10:22.683435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-03f95d78-5e8f-4a29-a631-e43b02867154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:22.769270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:22.805447 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f95d78-5e8f-4a29-a631-e43b02867154 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:10:22.886231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73871d75-fd41-4b9f-bacb-3de782f44f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:10:22.886832 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73871d75-fd41-4b9f-bacb-3de782f44f87 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:22.887097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73871d75-fd41-4b9f-bacb-3de782f44f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:22.887354 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73871d75-fd41-4b9f-bacb-3de782f44f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:22.888959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-73871d75-fd41-4b9f-bacb-3de782f44f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:23.002876 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73871d75-fd41-4b9f-bacb-3de782f44f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:10:23.004304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73871d75-fd41-4b9f-bacb-3de782f44f87 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:10:23.022735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 669/1340] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:10:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:10:27.318614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fbc48-0458-4e43-a9a5-c394b91093dd None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:27.319171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fbc48-0458-4e43-a9a5-c394b91093dd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:27.319291 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fbc48-0458-4e43-a9a5-c394b91093dd None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:27.321310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 672/1341] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:27.326858 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b489c-d760-4055-b696-bac4e0a28c1e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:27.327317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b489c-d760-4055-b696-bac4e0a28c1e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:27.327399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b489c-d760-4055-b696-bac4e0a28c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:27.332953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:27.402813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3a8b489c-d760-4055-b696-bac4e0a28c1e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:27.404033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3a8b489c-d760-4055-b696-bac4e0a28c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:27.512906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 670/1342] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:27.792738 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fbc48-0458-4e43-a9a5-c394b91093dd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:10:27.794350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e07fbc48-0458-4e43-a9a5-c394b91093dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:27.848109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:27.906588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fbc48-0458-4e43-a9a5-c394b91093dd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:10:27.986066 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c864712-b1e0-4709-a594-8388e6fc9d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:10:27.986674 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c864712-b1e0-4709-a594-8388e6fc9d57 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:27.986948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c864712-b1e0-4709-a594-8388e6fc9d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:27.987225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c864712-b1e0-4709-a594-8388e6fc9d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:27.988839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3c864712-b1e0-4709-a594-8388e6fc9d57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:28.088688 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:28.112196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c864712-b1e0-4709-a594-8388e6fc9d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:10:28.113650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c864712-b1e0-4709-a594-8388e6fc9d57 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:10:28.142892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 673/1343] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:10:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:10:30.927414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228ebb7-b9f0-4611-bb77-dfb0ce716207 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:30.928009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228ebb7-b9f0-4611-bb77-dfb0ce716207 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:30.928009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228ebb7-b9f0-4611-bb77-dfb0ce716207 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:30.929375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 671/1344] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:30.934719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dce45-1527-49c6-9173-09ebcda66039 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:30.935222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dce45-1527-49c6-9173-09ebcda66039 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:30.935222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5dce45-1527-49c6-9173-09ebcda66039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:31.008020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-fc5dce45-1527-49c6-9173-09ebcda66039 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:31.009256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fc5dce45-1527-49c6-9173-09ebcda66039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:31.117367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 674/1345] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:31.404775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228ebb7-b9f0-4611-bb77-dfb0ce716207 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:10:31.406333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3228ebb7-b9f0-4611-bb77-dfb0ce716207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:31.502397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:31.526336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228ebb7-b9f0-4611-bb77-dfb0ce716207 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:10:31.605877 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47457d30-91c6-4419-b6ac-adae62cba638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:10:31.606382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47457d30-91c6-4419-b6ac-adae62cba638 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:31.606647 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47457d30-91c6-4419-b6ac-adae62cba638 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:31.606903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47457d30-91c6-4419-b6ac-adae62cba638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:31.608473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-47457d30-91c6-4419-b6ac-adae62cba638 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:31.724455 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47457d30-91c6-4419-b6ac-adae62cba638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:10:31.726569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47457d30-91c6-4419-b6ac-adae62cba638 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:10:31.745315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 672/1346] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:10:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:10:36.155988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea102ebf-a0b6-4013-8a1f-f09c1dbbafe8 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:36.156480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea102ebf-a0b6-4013-8a1f-f09c1dbbafe8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:36.156480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea102ebf-a0b6-4013-8a1f-f09c1dbbafe8 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:36.158198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 675/1347] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:36.165049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b402d-99fa-4cdb-9cd7-4a4dcfbc617e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:36.165637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b402d-99fa-4cdb-9cd7-4a4dcfbc617e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:36.165637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b402d-99fa-4cdb-9cd7-4a4dcfbc617e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:36.182926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:36.240812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f78b402d-99fa-4cdb-9cd7-4a4dcfbc617e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:36.242424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f78b402d-99fa-4cdb-9cd7-4a4dcfbc617e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:36.354134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 673/1348] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:10:36.645992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea102ebf-a0b6-4013-8a1f-f09c1dbbafe8 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:10:36.648198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ea102ebf-a0b6-4013-8a1f-f09c1dbbafe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:36.714844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:36.763785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea102ebf-a0b6-4013-8a1f-f09c1dbbafe8 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:10:36.848295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd1a0b-9b76-4ee7-a2dc-9d263c72cf9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:10:36.848951 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd1a0b-9b76-4ee7-a2dc-9d263c72cf9a service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:36.849222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd1a0b-9b76-4ee7-a2dc-9d263c72cf9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:36.849486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd1a0b-9b76-4ee7-a2dc-9d263c72cf9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:36.851120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d6bd1a0b-9b76-4ee7-a2dc-9d263c72cf9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:36.963707 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:36.967767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd1a0b-9b76-4ee7-a2dc-9d263c72cf9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:10:36.969326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd1a0b-9b76-4ee7-a2dc-9d263c72cf9a service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:10:36.992785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 676/1349] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:10:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:10:41.897308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e376048-de4d-4971-ab07-52a1a2d49fb3 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:41.897308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e376048-de4d-4971-ab07-52a1a2d49fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:41.897308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e376048-de4d-4971-ab07-52a1a2d49fb3 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:41.906780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 674/1350] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:10:41 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:41.912623 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912cc2c-b5c8-4aa3-8a9e-d4d84ccfc017 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:41.913085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912cc2c-b5c8-4aa3-8a9e-d4d84ccfc017 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:41.914551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912cc2c-b5c8-4aa3-8a9e-d4d84ccfc017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:42.023551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f912cc2c-b5c8-4aa3-8a9e-d4d84ccfc017 None None] MFA Rules not processed for user `2b8be1ec4a7a45b88d84bd9c48fab407`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:42.025804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f912cc2c-b5c8-4aa3-8a9e-d4d84ccfc017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:42.179795 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 677/1351] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:10:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:46.350844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=59 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:46.395527 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b4b62-ed5a-4081-bea7-62792209cbbf None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:46.396784 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b4b62-ed5a-4081-bea7-62792209cbbf None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:46.397153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b4b62-ed5a-4081-bea7-62792209cbbf None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:46.399195 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 675/1352] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:10:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:46.404619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5981bde-7135-47f2-883c-f9b8b842ef98 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:46.405130 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5981bde-7135-47f2-883c-f9b8b842ef98 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:46.405244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5981bde-7135-47f2-883c-f9b8b842ef98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:46.464104 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:46.489067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a5981bde-7135-47f2-883c-f9b8b842ef98 None None] MFA Rules not processed for user `2b8be1ec4a7a45b88d84bd9c48fab407`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:46.490373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a5981bde-7135-47f2-883c-f9b8b842ef98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:46.612942 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 678/1353] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:10:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:48.034613 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c87c57-6da5-49b2-a4ed-ab90625aa828 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:48.035230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c87c57-6da5-49b2-a4ed-ab90625aa828 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:48.035230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c87c57-6da5-49b2-a4ed-ab90625aa828 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:48.036721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 676/1354] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:48.042102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f68ae-8aa5-4f0f-8fc8-9d273960328a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:48.042624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f68ae-8aa5-4f0f-8fc8-9d273960328a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:48.042624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f68ae-8aa5-4f0f-8fc8-9d273960328a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:48.124487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-542f68ae-8aa5-4f0f-8fc8-9d273960328a None None] MFA Rules not processed for user `2b8be1ec4a7a45b88d84bd9c48fab407`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:48.125693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-542f68ae-8aa5-4f0f-8fc8-9d273960328a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:48.172579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:48.249160 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 679/1355] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:10:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:51.702404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8aebd-4566-448c-bcce-b8b524dc7548 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:51.703228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8aebd-4566-448c-bcce-b8b524dc7548 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:51.703645 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8aebd-4566-448c-bcce-b8b524dc7548 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:51.705484 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 677/1356] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:51.711178 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598b822-cfe2-4814-ab1a-499e3e922cf1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:51.712065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598b822-cfe2-4814-ab1a-499e3e922cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:51.713331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598b822-cfe2-4814-ab1a-499e3e922cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:51.798916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f598b822-cfe2-4814-ab1a-499e3e922cf1 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:51.801072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f598b822-cfe2-4814-ab1a-499e3e922cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:51.915805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 680/1357] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:52.216670 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30ada6-a69e-4981-bff7-213afe3c09b2 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:52.217059 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30ada6-a69e-4981-bff7-213afe3c09b2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:52.217462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30ada6-a69e-4981-bff7-213afe3c09b2 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:52.218832 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 678/1358] 10.222.0.75 () {54 vars in 961 bytes} [Fri Sep 23 15:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:52.228474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560019a-673d-43e2-a0a6-b0f82d75032b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:52.229727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560019a-673d-43e2-a0a6-b0f82d75032b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:52.230594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d560019a-673d-43e2-a0a6-b0f82d75032b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:52.257449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:52.322919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d560019a-673d-43e2-a0a6-b0f82d75032b None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:52.324265 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d560019a-673d-43e2-a0a6-b0f82d75032b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:52.458443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 681/1359] 10.222.0.75 () {58 vars in 1058 bytes} [Fri Sep 23 15:10:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:52.467003 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30ada6-a69e-4981-bff7-213afe3c09b2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:10:52.468551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6c30ada6-a69e-4981-bff7-213afe3c09b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:52.633162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30ada6-a69e-4981-bff7-213afe3c09b2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:10:52.707623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:52.751814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb08bfb-6a48-4ea3-88a2-6bf60ad5fc53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:10:52.752430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb08bfb-6a48-4ea3-88a2-6bf60ad5fc53 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:52.752791 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb08bfb-6a48-4ea3-88a2-6bf60ad5fc53 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:52.753118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb08bfb-6a48-4ea3-88a2-6bf60ad5fc53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:52.754796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9eb08bfb-6a48-4ea3-88a2-6bf60ad5fc53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:52.846434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb08bfb-6a48-4ea3-88a2-6bf60ad5fc53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:10:52.848134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb08bfb-6a48-4ea3-88a2-6bf60ad5fc53 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:10:52.868205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 679/1360] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:10:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:10:55.250886 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e49c7-8ab2-47c3-b362-a01dd3c546b9 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:55.251418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e49c7-8ab2-47c3-b362-a01dd3c546b9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:55.251418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e49c7-8ab2-47c3-b362-a01dd3c546b9 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:55.252910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 682/1361] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:55.258513 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c76741-5a81-43f1-9fb9-d3e330765bfc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:55.258998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c76741-5a81-43f1-9fb9-d3e330765bfc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:55.258998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c76741-5a81-43f1-9fb9-d3e330765bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:55.340426 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-98c76741-5a81-43f1-9fb9-d3e330765bfc None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:55.341333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-98c76741-5a81-43f1-9fb9-d3e330765bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:55.383520 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:55.447315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 680/1362] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:55.733053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e49c7-8ab2-47c3-b362-a01dd3c546b9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:10:55.734602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b87e49c7-8ab2-47c3-b362-a01dd3c546b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:55.806055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:55.891759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e49c7-8ab2-47c3-b362-a01dd3c546b9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:10:56.031154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc3c6e-4782-420a-a51b-6c52973cefde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:10:56.031758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc3c6e-4782-420a-a51b-6c52973cefde service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:56.032061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc3c6e-4782-420a-a51b-6c52973cefde service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:56.032287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc3c6e-4782-420a-a51b-6c52973cefde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:56.034029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dffc3c6e-4782-420a-a51b-6c52973cefde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:56.059953 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:56.133400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc3c6e-4782-420a-a51b-6c52973cefde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:10:56.135027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc3c6e-4782-420a-a51b-6c52973cefde service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:10:56.155658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 683/1363] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:10:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:10:58.666177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a79e6-2f6d-48e6-96eb-0a117ece8338 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:58.666744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a79e6-2f6d-48e6-96eb-0a117ece8338 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:58.666744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a79e6-2f6d-48e6-96eb-0a117ece8338 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:58.668265 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 681/1364] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:58.674192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4ba67-a8d9-41c2-8c01-0ce5ddeccc4f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:58.675122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4ba67-a8d9-41c2-8c01-0ce5ddeccc4f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:58.675392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4ba67-a8d9-41c2-8c01-0ce5ddeccc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:58.749082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2ff4ba67-a8d9-41c2-8c01-0ce5ddeccc4f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:58.750243 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2ff4ba67-a8d9-41c2-8c01-0ce5ddeccc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:58.849755 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:58.860629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 684/1365] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:59.160676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363f22a-f43c-4cba-8b89-1acaa99a3265 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:59.160902 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363f22a-f43c-4cba-8b89-1acaa99a3265 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:59.161237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363f22a-f43c-4cba-8b89-1acaa99a3265 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:59.162746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 682/1366] 10.222.0.75 () {54 vars in 961 bytes} [Fri Sep 23 15:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:10:59.171460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848bf33-4582-4bfa-8ed3-f7e6aa2f17ad None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:59.171977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848bf33-4582-4bfa-8ed3-f7e6aa2f17ad None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:59.172255 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848bf33-4582-4bfa-8ed3-f7e6aa2f17ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:59.267916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5848bf33-4582-4bfa-8ed3-f7e6aa2f17ad None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:10:59.269143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5848bf33-4582-4bfa-8ed3-f7e6aa2f17ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:59.427018 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 685/1367] 10.222.0.75 () {58 vars in 1058 bytes} [Fri Sep 23 15:10:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:10:59.436055 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363f22a-f43c-4cba-8b89-1acaa99a3265 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:10:59.438502 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d363f22a-f43c-4cba-8b89-1acaa99a3265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:59.661678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363f22a-f43c-4cba-8b89-1acaa99a3265 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:10:59.687974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:10:59.828375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7fbb4-3ea8-4a87-96df-44fa2174b43a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:10:59.829434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7fbb4-3ea8-4a87-96df-44fa2174b43a service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:59.829921 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7fbb4-3ea8-4a87-96df-44fa2174b43a service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:59.830375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7fbb4-3ea8-4a87-96df-44fa2174b43a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:59.833633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-68a7fbb4-3ea8-4a87-96df-44fa2174b43a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:10:59.947241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaee6b1-ec9f-42bd-9475-1b3b172d006b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:59.947808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaee6b1-ec9f-42bd-9475-1b3b172d006b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:59.947808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaee6b1-ec9f-42bd-9475-1b3b172d006b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:59.949391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 686/1368] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:10:59.954637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c68110-7275-4e8a-8a77-a69be7948663 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:10:59.954962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c68110-7275-4e8a-8a77-a69be7948663 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:10:59.955169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c68110-7275-4e8a-8a77-a69be7948663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:10:59.961956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7fbb4-3ea8-4a87-96df-44fa2174b43a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:10:59.964381 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7fbb4-3ea8-4a87-96df-44fa2174b43a service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:10:59.987064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 683/1369] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:10:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:00.020360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:00.030465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f5c68110-7275-4e8a-8a77-a69be7948663 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:00.031761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f5c68110-7275-4e8a-8a77-a69be7948663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:00.189320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 687/1370] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:11:00.526450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c0dca-97af-422d-9122-a1095a528799 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:00.526450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c0dca-97af-422d-9122-a1095a528799 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:00.526450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c0dca-97af-422d-9122-a1095a528799 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:00.528395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 684/1371] 10.222.0.75 () {54 vars in 963 bytes} [Fri Sep 23 15:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:00.532086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdffa9a-3c09-4695-91f8-b34bcf501a8e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:00.532247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdffa9a-3c09-4695-91f8-b34bcf501a8e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:00.532440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdffa9a-3c09-4695-91f8-b34bcf501a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:00.614877 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ffdffa9a-3c09-4695-91f8-b34bcf501a8e None None] MFA Rules not processed for user `1cd51776db3d426ea1bf1133344fa187`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:00.616398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ffdffa9a-3c09-4695-91f8-b34bcf501a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:00.691405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:00.700745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 688/1372] 10.222.0.75 () {58 vars in 1060 bytes} [Fri Sep 23 15:11:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:11:00.708353 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c0dca-97af-422d-9122-a1095a528799 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:00.710183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1d7c0dca-97af-422d-9122-a1095a528799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:00.742870 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:00.828601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c0dca-97af-422d-9122-a1095a528799 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:00.906380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe6b53-874b-4eb3-9d54-ce83f54cb974 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:00.907006 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe6b53-874b-4eb3-9d54-ce83f54cb974 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:00.907274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe6b53-874b-4eb3-9d54-ce83f54cb974 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:00.907535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe6b53-874b-4eb3-9d54-ce83f54cb974 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:00.909174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9cbe6b53-874b-4eb3-9d54-ce83f54cb974 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:01.017219 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:01.061610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe6b53-874b-4eb3-9d54-ce83f54cb974 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:01.063461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe6b53-874b-4eb3-9d54-ce83f54cb974 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:01.088807 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 685/1373] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:02.262898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e8b71-63b2-43fa-8a27-e97dc2fe2be5 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:02.263495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e8b71-63b2-43fa-8a27-e97dc2fe2be5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:02.263597 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e8b71-63b2-43fa-8a27-e97dc2fe2be5 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:02.264947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 689/1374] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:02.270520 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1c1fc-fbe5-4e70-a68d-c17f2d92ebf9 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:02.270926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1c1fc-fbe5-4e70-a68d-c17f2d92ebf9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:02.271084 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1c1fc-fbe5-4e70-a68d-c17f2d92ebf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:02.346490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2cc1c1fc-fbe5-4e70-a68d-c17f2d92ebf9 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:02.347713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2cc1c1fc-fbe5-4e70-a68d-c17f2d92ebf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:02.453332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 686/1375] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:02.744104 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e8b71-63b2-43fa-8a27-e97dc2fe2be5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:02.745776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6b4e8b71-63b2-43fa-8a27-e97dc2fe2be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:02.901830 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e8b71-63b2-43fa-8a27-e97dc2fe2be5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:02.972309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:03.022729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02fa85-bceb-4f71-b86c-a11a628d27c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:03.024038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02fa85-bceb-4f71-b86c-a11a628d27c9 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:03.024309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02fa85-bceb-4f71-b86c-a11a628d27c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:03.024576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02fa85-bceb-4f71-b86c-a11a628d27c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:03.026218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3a02fa85-bceb-4f71-b86c-a11a628d27c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:03.123516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a02fa85-bceb-4f71-b86c-a11a628d27c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:03.125334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a02fa85-bceb-4f71-b86c-a11a628d27c9 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:03.145166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 690/1376] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:05.361885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1733f56-9ab4-4c6e-ae0f-cc5840904fb3 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:05.362461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1733f56-9ab4-4c6e-ae0f-cc5840904fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:05.362652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1733f56-9ab4-4c6e-ae0f-cc5840904fb3 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:05.364295 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 687/1377] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:05.370206 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f421692-ff5c-45d0-a6ac-be45b7562dcc None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:05.370653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f421692-ff5c-45d0-a6ac-be45b7562dcc None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:05.370734 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f421692-ff5c-45d0-a6ac-be45b7562dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:05.444257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4f421692-ff5c-45d0-a6ac-be45b7562dcc None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:05.445628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4f421692-ff5c-45d0-a6ac-be45b7562dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:05.478593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:05.556309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 691/1378] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:11:05.839041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1733f56-9ab4-4c6e-ae0f-cc5840904fb3 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:05.840959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b1733f56-9ab4-4c6e-ae0f-cc5840904fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:05.882802 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:06.001008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1733f56-9ab4-4c6e-ae0f-cc5840904fb3 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:06.136490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:06.142384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754668f-c781-46aa-aa81-13f9e436e37b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:06.143004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754668f-c781-46aa-aa81-13f9e436e37b service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:06.143279 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754668f-c781-46aa-aa81-13f9e436e37b service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:06.143534 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754668f-c781-46aa-aa81-13f9e436e37b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:06.145552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4754668f-c781-46aa-aa81-13f9e436e37b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:06.256353 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4754668f-c781-46aa-aa81-13f9e436e37b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:06.258377 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4754668f-c781-46aa-aa81-13f9e436e37b service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:06.278674 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 688/1379] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:08.488328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e19302-1d93-407b-bf27-5f2ca8bab31c None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:08.489604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e19302-1d93-407b-bf27-5f2ca8bab31c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:08.489924 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e19302-1d93-407b-bf27-5f2ca8bab31c None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:08.492477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 692/1380] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:11:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:08.498686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5ba0d-33aa-4218-a2be-e428220a2201 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:08.499101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5ba0d-33aa-4218-a2be-e428220a2201 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:08.499260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c5ba0d-33aa-4218-a2be-e428220a2201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:08.572160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-42c5ba0d-33aa-4218-a2be-e428220a2201 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:08.574327 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-42c5ba0d-33aa-4218-a2be-e428220a2201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:08.645051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:08.678608 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 689/1381] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:08.963317 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e19302-1d93-407b-bf27-5f2ca8bab31c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:08.964984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-28e19302-1d93-407b-bf27-5f2ca8bab31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:09.144396 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e19302-1d93-407b-bf27-5f2ca8bab31c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:09.158088 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:09.260537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38825da-e188-4ca2-8d2b-2a4071f6dd10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:09.261082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38825da-e188-4ca2-8d2b-2a4071f6dd10 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:09.261350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38825da-e188-4ca2-8d2b-2a4071f6dd10 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:09.261612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38825da-e188-4ca2-8d2b-2a4071f6dd10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:09.263211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a38825da-e188-4ca2-8d2b-2a4071f6dd10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:09.361996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38825da-e188-4ca2-8d2b-2a4071f6dd10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:09.363578 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38825da-e188-4ca2-8d2b-2a4071f6dd10 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:09.382261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 693/1382] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:11.599955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e7a43-a557-40f9-94cf-263b5346857a None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:11.600522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e7a43-a557-40f9-94cf-263b5346857a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:11.600522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e7a43-a557-40f9-94cf-263b5346857a None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:11.602221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 690/1383] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:11.608107 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cf7de-36a2-4eb1-8efc-aad7c0bfc2d6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:11.608558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cf7de-36a2-4eb1-8efc-aad7c0bfc2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:11.608843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cf7de-36a2-4eb1-8efc-aad7c0bfc2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:11.638005 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:11.689640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5d0cf7de-36a2-4eb1-8efc-aad7c0bfc2d6 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:11.691481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5d0cf7de-36a2-4eb1-8efc-aad7c0bfc2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:11.815304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 694/1384] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:12.098338 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e7a43-a557-40f9-94cf-263b5346857a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:12.099917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-132e7a43-a557-40f9-94cf-263b5346857a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:12.279542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132e7a43-a557-40f9-94cf-263b5346857a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:12.342136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:12.409184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02442cf-bbfc-4eb9-84b0-41b815afaa3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:12.409799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02442cf-bbfc-4eb9-84b0-41b815afaa3e service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:12.410084 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02442cf-bbfc-4eb9-84b0-41b815afaa3e service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:12.410349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02442cf-bbfc-4eb9-84b0-41b815afaa3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:12.411971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d02442cf-bbfc-4eb9-84b0-41b815afaa3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:12.508697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02442cf-bbfc-4eb9-84b0-41b815afaa3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:12.510408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02442cf-bbfc-4eb9-84b0-41b815afaa3e service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:12.529342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 691/1385] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:14.933937 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47382188-9261-47a8-98ac-0c6c20df187b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:14.934527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47382188-9261-47a8-98ac-0c6c20df187b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:14.934527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47382188-9261-47a8-98ac-0c6c20df187b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:14.935960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 695/1386] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:14.941524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b472e-5c97-4e16-bf40-c9eb5d69d710 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:14.941954 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b472e-5c97-4e16-bf40-c9eb5d69d710 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:14.942046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b472e-5c97-4e16-bf40-c9eb5d69d710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:15.013336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0d7b472e-5c97-4e16-bf40-c9eb5d69d710 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:15.016973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0d7b472e-5c97-4e16-bf40-c9eb5d69d710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:15.024792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:15.119916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 692/1387] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:15.412020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47382188-9261-47a8-98ac-0c6c20df187b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:15.413636 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-47382188-9261-47a8-98ac-0c6c20df187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:15.512927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:15.571699 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47382188-9261-47a8-98ac-0c6c20df187b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:15.689707 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee36fd-dbbb-42a6-955c-e5404cbe7ea4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:15.690235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee36fd-dbbb-42a6-955c-e5404cbe7ea4 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:15.690512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee36fd-dbbb-42a6-955c-e5404cbe7ea4 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:15.690778 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee36fd-dbbb-42a6-955c-e5404cbe7ea4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:15.692470 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-72ee36fd-dbbb-42a6-955c-e5404cbe7ea4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:15.754548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:15.792124 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee36fd-dbbb-42a6-955c-e5404cbe7ea4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:15.793746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee36fd-dbbb-42a6-955c-e5404cbe7ea4 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:15.813478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 696/1388] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:16.391744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d12485-5c31-44e8-8233-a103410ee910 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:16.392333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d12485-5c31-44e8-8233-a103410ee910 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:16.392333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d12485-5c31-44e8-8233-a103410ee910 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:16.393759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 693/1389] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:16.399348 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304dde3-e038-48d8-a097-b43e592d0050 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:16.399836 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304dde3-e038-48d8-a097-b43e592d0050 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:16.399925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304dde3-e038-48d8-a097-b43e592d0050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:16.471727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-3304dde3-e038-48d8-a097-b43e592d0050 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:16.472980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3304dde3-e038-48d8-a097-b43e592d0050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:16.582811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 697/1390] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:16.886047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc53561-23cf-4461-8ddb-7f708e756272 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:16.886331 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc53561-23cf-4461-8ddb-7f708e756272 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:16.886601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc53561-23cf-4461-8ddb-7f708e756272 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:16.888001 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 694/1391] 10.222.0.75 () {54 vars in 962 bytes} [Fri Sep 23 15:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:16.893350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7d504-dc21-471c-b7a7-9b7de8fc20f4 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:16.893712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7d504-dc21-471c-b7a7-9b7de8fc20f4 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:16.894053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7d504-dc21-471c-b7a7-9b7de8fc20f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:16.941652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:16.973629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-40c7d504-dc21-471c-b7a7-9b7de8fc20f4 None None] MFA Rules not processed for user `ac783fbe95ea4b74bd46af415c9734fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:16.974832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-40c7d504-dc21-471c-b7a7-9b7de8fc20f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:17.056719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 698/1392] 10.222.0.75 () {58 vars in 1059 bytes} [Fri Sep 23 15:11:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:17.065984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc53561-23cf-4461-8ddb-7f708e756272 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:17.067524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dcc53561-23cf-4461-8ddb-7f708e756272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:17.178399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc53561-23cf-4461-8ddb-7f708e756272 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:17.217862 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:17.255817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554ee2a-c267-4bc0-9b8f-63d0fd948ec8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac783fbe95ea4b74bd46af415c9734fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:17.256421 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ee2a-c267-4bc0-9b8f-63d0fd948ec8 service glance] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:17.256698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ee2a-c267-4bc0-9b8f-63d0fd948ec8 service glance] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:17.256987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ee2a-c267-4bc0-9b8f-63d0fd948ec8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:17.258607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f554ee2a-c267-4bc0-9b8f-63d0fd948ec8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:17.369507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554ee2a-c267-4bc0-9b8f-63d0fd948ec8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:17.370962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554ee2a-c267-4bc0-9b8f-63d0fd948ec8 service glance] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:17.389046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 695/1393] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:11:18.089839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeec465-140a-4937-9eaf-f73c89862742 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:18.090411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeec465-140a-4937-9eaf-f73c89862742 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:18.090411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeec465-140a-4937-9eaf-f73c89862742 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:18.091730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 699/1394] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:18.097516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62999e-22eb-48e0-a955-bd422f04ac25 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:18.098080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62999e-22eb-48e0-a955-bd422f04ac25 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:18.098080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62999e-22eb-48e0-a955-bd422f04ac25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:18.154957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae886a37-5db6-4aa2-b236-835ebc987cac None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:18.155276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae886a37-5db6-4aa2-b236-835ebc987cac None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:18.155528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae886a37-5db6-4aa2-b236-835ebc987cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:18.167534 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2a62999e-22eb-48e0-a955-bd422f04ac25 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:18.168697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2a62999e-22eb-48e0-a955-bd422f04ac25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:18.187469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:18.231129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ae886a37-5db6-4aa2-b236-835ebc987cac None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:18.239205 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ae886a37-5db6-4aa2-b236-835ebc987cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:18.284515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 696/1395] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:18.325526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:18.333336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 700/1396] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:11:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:11:18.375394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc80b61-8bdc-4ff1-bc68-e62eeeaccd0f None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:18.375676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc80b61-8bdc-4ff1-bc68-e62eeeaccd0f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:18.375946 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc80b61-8bdc-4ff1-bc68-e62eeeaccd0f None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:18.377638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 697/1397] 10.222.0.75 () {54 vars in 943 bytes} [Fri Sep 23 15:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:18.383702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a14fb-c848-41f0-9d08-5b1b585e8669 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:18.383984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a14fb-c848-41f0-9d08-5b1b585e8669 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:18.384252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a14fb-c848-41f0-9d08-5b1b585e8669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:18.461763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f20a14fb-c848-41f0-9d08-5b1b585e8669 None None] MFA Rules not processed for user `a8614f90cebb4a4eaf7ab5512927fb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:18.462902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f20a14fb-c848-41f0-9d08-5b1b585e8669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:18.579274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 701/1398] 10.222.0.75 () {58 vars in 1040 bytes} [Fri Sep 23 15:11:18 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:18.581966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc80b61-8bdc-4ff1-bc68-e62eeeaccd0f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:18.583519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7bc80b61-8bdc-4ff1-bc68-e62eeeaccd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:18.588944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a14fb-c848-41f0-9d08-5b1b585e8669 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:18.598578 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a14fb-c848-41f0-9d08-5b1b585e8669 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:18.650716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:18.709640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c3a1e-e09d-4cfd-9b9c-aeb97c158458 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:18.710380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c3a1e-e09d-4cfd-9b9c-aeb97c158458 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:18.711199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c3a1e-e09d-4cfd-9b9c-aeb97c158458 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:18.711474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c3a1e-e09d-4cfd-9b9c-aeb97c158458 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:18.713253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-416c3a1e-e09d-4cfd-9b9c-aeb97c158458 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:18.732096 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc80b61-8bdc-4ff1-bc68-e62eeeaccd0f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:18.734918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:18.800125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c3a1e-e09d-4cfd-9b9c-aeb97c158458 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:18.801801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c3a1e-e09d-4cfd-9b9c-aeb97c158458 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:18.809989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 702/1399] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:11:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:11:18.852554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74230b26-a619-4133-acb9-1694e225b1c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:18.853646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230b26-a619-4133-acb9-1694e225b1c3 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:18.853646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230b26-a619-4133-acb9-1694e225b1c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:18.853796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74230b26-a619-4133-acb9-1694e225b1c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:18.855391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-74230b26-a619-4133-acb9-1694e225b1c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:18.951944 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74230b26-a619-4133-acb9-1694e225b1c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:18.953584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74230b26-a619-4133-acb9-1694e225b1c3 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:18.974386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 698/1400] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:21.230488 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516334db-bcfb-412c-95ff-3859fdd8b091 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:21.231101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516334db-bcfb-412c-95ff-3859fdd8b091 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:21.231101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516334db-bcfb-412c-95ff-3859fdd8b091 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:21.232393 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 703/1401] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:21.237835 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8b80b-e160-4d77-a175-f012d926e8b7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:21.238268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8b80b-e160-4d77-a175-f012d926e8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:21.238364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb8b80b-e160-4d77-a175-f012d926e8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:21.248847 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:21.310880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1fb8b80b-e160-4d77-a175-f012d926e8b7 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:21.312107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1fb8b80b-e160-4d77-a175-f012d926e8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:21.422441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 699/1402] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:21.717618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516334db-bcfb-412c-95ff-3859fdd8b091 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:21.719226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-516334db-bcfb-412c-95ff-3859fdd8b091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:21.808973 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:21.883966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516334db-bcfb-412c-95ff-3859fdd8b091 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:22.010245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0760fb7-858e-4000-a503-2a45e50080c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:22.011149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0760fb7-858e-4000-a503-2a45e50080c3 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:22.011423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0760fb7-858e-4000-a503-2a45e50080c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:22.011690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0760fb7-858e-4000-a503-2a45e50080c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:22.013462 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e0760fb7-858e-4000-a503-2a45e50080c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:22.064784 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:22.117418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0760fb7-858e-4000-a503-2a45e50080c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:22.119013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0760fb7-858e-4000-a503-2a45e50080c3 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:22.139078 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 704/1403] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:24.370537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539678e-feaa-4ab3-b208-b26a89359e69 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:24.371126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539678e-feaa-4ab3-b208-b26a89359e69 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:24.371126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539678e-feaa-4ab3-b208-b26a89359e69 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:24.372578 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 700/1404] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:24.378424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93057142-198c-46a2-808e-cbdf5663ec58 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:24.378861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93057142-198c-46a2-808e-cbdf5663ec58 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:24.379094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93057142-198c-46a2-808e-cbdf5663ec58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:24.452799 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-93057142-198c-46a2-808e-cbdf5663ec58 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:24.454100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-93057142-198c-46a2-808e-cbdf5663ec58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:24.570856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 705/1405] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:24.856960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539678e-feaa-4ab3-b208-b26a89359e69 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:24.858798 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8539678e-feaa-4ab3-b208-b26a89359e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:24.958348 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:25.030272 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539678e-feaa-4ab3-b208-b26a89359e69 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:25.158221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa04900-579a-444e-864e-bfb77071a509 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:25.158854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa04900-579a-444e-864e-bfb77071a509 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:25.159131 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa04900-579a-444e-864e-bfb77071a509 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:25.159402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa04900-579a-444e-864e-bfb77071a509 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:25.161041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3aa04900-579a-444e-864e-bfb77071a509 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:25.218291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:25.269142 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa04900-579a-444e-864e-bfb77071a509 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:25.270774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa04900-579a-444e-864e-bfb77071a509 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:25.290159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 701/1406] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:27.588100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb83f82-2a50-492d-a345-36a22d27fb4a None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:27.588854 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb83f82-2a50-492d-a345-36a22d27fb4a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:27.588854 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb83f82-2a50-492d-a345-36a22d27fb4a None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:27.590031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 706/1407] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:27.595605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a605f8-1ab7-46d0-807b-7d46edb4c27d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:27.596101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a605f8-1ab7-46d0-807b-7d46edb4c27d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:27.596194 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a605f8-1ab7-46d0-807b-7d46edb4c27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:27.682673 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-21a605f8-1ab7-46d0-807b-7d46edb4c27d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:27.683937 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-21a605f8-1ab7-46d0-807b-7d46edb4c27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:27.791848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 702/1408] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:28.076802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb83f82-2a50-492d-a345-36a22d27fb4a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:28.080318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1eb83f82-2a50-492d-a345-36a22d27fb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:28.102559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:28.254529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb83f82-2a50-492d-a345-36a22d27fb4a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:28.352918 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:28.379473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337d3b4-2520-4165-a5d6-bddd1e733181 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:28.380085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337d3b4-2520-4165-a5d6-bddd1e733181 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:28.380358 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337d3b4-2520-4165-a5d6-bddd1e733181 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:28.380655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337d3b4-2520-4165-a5d6-bddd1e733181 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:28.382331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2337d3b4-2520-4165-a5d6-bddd1e733181 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:28.486211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337d3b4-2520-4165-a5d6-bddd1e733181 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:28.487872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337d3b4-2520-4165-a5d6-bddd1e733181 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:28.507056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 707/1409] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:37.790255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:38.504576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=54 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:41.916496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d61ee9-1629-4d32-afa5-9d0ba5e24eb6 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:41.917198 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d61ee9-1629-4d32-afa5-9d0ba5e24eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:41.917198 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d61ee9-1629-4d32-afa5-9d0ba5e24eb6 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:41.918519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 703/1410] 10.222.0.75 () {54 vars in 918 bytes} [Fri Sep 23 15:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:41.924809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5488e-4df5-44e5-a05e-7b1669882c42 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:41.925283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5488e-4df5-44e5-a05e-7b1669882c42 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:41.925385 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5488e-4df5-44e5-a05e-7b1669882c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:42.013590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-21a5488e-4df5-44e5-a05e-7b1669882c42 None None] MFA Rules not processed for user `2b8be1ec4a7a45b88d84bd9c48fab407`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:42.014846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-21a5488e-4df5-44e5-a05e-7b1669882c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:42.128854 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 708/1411] 10.222.0.75 () {58 vars in 1015 bytes} [Fri Sep 23 15:11:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:42.145640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be32552-c6d6-4e11-96b0-5cb594b13fea None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:42.145914 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be32552-c6d6-4e11-96b0-5cb594b13fea None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:42.146177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be32552-c6d6-4e11-96b0-5cb594b13fea None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:42.147535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 704/1412] 10.222.0.75 () {54 vars in 918 bytes} [Fri Sep 23 15:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:42.153100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff36cb5-1752-49bd-a99d-653c766bd357 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:42.153378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff36cb5-1752-49bd-a99d-653c766bd357 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:42.153648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff36cb5-1752-49bd-a99d-653c766bd357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:42.239022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:42.242201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0ff36cb5-1752-49bd-a99d-653c766bd357 None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:42.244148 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0ff36cb5-1752-49bd-a99d-653c766bd357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:42.383123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 709/1413] 10.222.0.75 () {58 vars in 1015 bytes} [Fri Sep 23 15:11:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:45.035230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda400a4-a2f3-46ce-92dc-30e6e8d836e9 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:45.035863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda400a4-a2f3-46ce-92dc-30e6e8d836e9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:45.035863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda400a4-a2f3-46ce-92dc-30e6e8d836e9 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:45.037155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:45.042846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49212b89-f958-4ae0-a404-3ce779020590 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:45.043256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49212b89-f958-4ae0-a404-3ce779020590 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:45.043459 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49212b89-f958-4ae0-a404-3ce779020590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:45.131079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-49212b89-f958-4ae0-a404-3ce779020590 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:45.132318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-49212b89-f958-4ae0-a404-3ce779020590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:45.199097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:45.244209 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:45.539466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda400a4-a2f3-46ce-92dc-30e6e8d836e9 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:45.541651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dda400a4-a2f3-46ce-92dc-30e6e8d836e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:45.728967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda400a4-a2f3-46ce-92dc-30e6e8d836e9 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:45.806263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:45.848532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3bf0e-fd5c-4159-9be3-55ed62997d59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:45.849159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3bf0e-fd5c-4159-9be3-55ed62997d59 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:45.849476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3bf0e-fd5c-4159-9be3-55ed62997d59 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:45.849728 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3bf0e-fd5c-4159-9be3-55ed62997d59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:45.851368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-42b3bf0e-fd5c-4159-9be3-55ed62997d59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:45.950782 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3bf0e-fd5c-4159-9be3-55ed62997d59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:45.952375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3bf0e-fd5c-4159-9be3-55ed62997d59 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:45.970339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 706/1416] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:47.145854 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac9f33-d764-45b2-be08-d477c82135a7 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:47.146426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac9f33-d764-45b2-be08-d477c82135a7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:47.146426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac9f33-d764-45b2-be08-d477c82135a7 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:47.147759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:47.154962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f9d99-fee3-4ed1-a2cb-a9649b8f0ea6 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:47.155472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f9d99-fee3-4ed1-a2cb-a9649b8f0ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:47.155472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f9d99-fee3-4ed1-a2cb-a9649b8f0ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:47.239524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e08f9d99-fee3-4ed1-a2cb-a9649b8f0ea6 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:47.241098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e08f9d99-fee3-4ed1-a2cb-a9649b8f0ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:47.268911 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:47.343272 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:47.521457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac9f33-d764-45b2-be08-d477c82135a7 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:47.522948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b9ac9f33-d764-45b2-be08-d477c82135a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:47.678804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac9f33-d764-45b2-be08-d477c82135a7 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:47.722254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:47.804814 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef081e5-34c9-4097-ae39-ed705eff3c0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:47.805434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef081e5-34c9-4097-ae39-ed705eff3c0e service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:47.805708 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef081e5-34c9-4097-ae39-ed705eff3c0e service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:47.805977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef081e5-34c9-4097-ae39-ed705eff3c0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:47.807804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9ef081e5-34c9-4097-ae39-ed705eff3c0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:47.911338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef081e5-34c9-4097-ae39-ed705eff3c0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:47.912868 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef081e5-34c9-4097-ae39-ed705eff3c0e service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:47.931183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 712/1419] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:49.112258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ff016-26a8-471d-9be2-af76fd8c0bdb None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:49.112865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ff016-26a8-471d-9be2-af76fd8c0bdb None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:49.112865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ff016-26a8-471d-9be2-af76fd8c0bdb None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:49.114177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:49.122099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1538e92-7bc4-4a50-94cc-39b1ed114a6e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:49.122875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1538e92-7bc4-4a50-94cc-39b1ed114a6e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:49.123028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1538e92-7bc4-4a50-94cc-39b1ed114a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:49.188823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:49.211547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e1538e92-7bc4-4a50-94cc-39b1ed114a6e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:49.212806 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e1538e92-7bc4-4a50-94cc-39b1ed114a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:49.321934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:49.478935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ff016-26a8-471d-9be2-af76fd8c0bdb None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:49.484538 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ab3ff016-26a8-471d-9be2-af76fd8c0bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:49.645594 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ff016-26a8-471d-9be2-af76fd8c0bdb None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:49.658840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:49.780031 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673eeb2-638a-48a5-8472-a527e4ad230d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:49.780649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673eeb2-638a-48a5-8472-a527e4ad230d service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:49.780956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673eeb2-638a-48a5-8472-a527e4ad230d service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:49.781214 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673eeb2-638a-48a5-8472-a527e4ad230d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:49.782832 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7673eeb2-638a-48a5-8472-a527e4ad230d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:49.880563 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673eeb2-638a-48a5-8472-a527e4ad230d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:49.882197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673eeb2-638a-48a5-8472-a527e4ad230d service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:49.900449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 709/1422] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:51.027126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02e565-83db-4789-af37-5f576fca5b8f None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:51.027770 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02e565-83db-4789-af37-5f576fca5b8f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:51.027770 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02e565-83db-4789-af37-5f576fca5b8f None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:51.029058 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:51.036579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1dce2-7196-4d9f-b6d3-03fbaaed2315 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:51.037431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1dce2-7196-4d9f-b6d3-03fbaaed2315 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:51.037558 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1dce2-7196-4d9f-b6d3-03fbaaed2315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:51.072154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:51.134024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-26b1dce2-7196-4d9f-b6d3-03fbaaed2315 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:51.135247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-26b1dce2-7196-4d9f-b6d3-03fbaaed2315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:51.236720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:51.390457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02e565-83db-4789-af37-5f576fca5b8f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:51.392976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6b02e565-83db-4789-af37-5f576fca5b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:51.515906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:51.552986 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02e565-83db-4789-af37-5f576fca5b8f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:51.677689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67245ef-107f-455b-9cfb-a9d1f411a334 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:51.678193 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67245ef-107f-455b-9cfb-a9d1f411a334 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:51.678455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67245ef-107f-455b-9cfb-a9d1f411a334 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:51.678716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67245ef-107f-455b-9cfb-a9d1f411a334 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:51.680292 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d67245ef-107f-455b-9cfb-a9d1f411a334 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:51.766553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:51.784110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67245ef-107f-455b-9cfb-a9d1f411a334 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:51.785724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67245ef-107f-455b-9cfb-a9d1f411a334 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:51.807355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 715/1425] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:52.938356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269b9fe-ac04-4790-a7c7-174550604e06 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:52.939085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269b9fe-ac04-4790-a7c7-174550604e06 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:52.939085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269b9fe-ac04-4790-a7c7-174550604e06 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:52.940298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:11:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:52.948898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31d22c-dff6-4bce-9c9c-17be7e06f780 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:52.949652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31d22c-dff6-4bce-9c9c-17be7e06f780 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:52.949821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df31d22c-dff6-4bce-9c9c-17be7e06f780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:53.035742 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-df31d22c-dff6-4bce-9c9c-17be7e06f780 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:53.037006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-df31d22c-dff6-4bce-9c9c-17be7e06f780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:53.147523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:53.315943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269b9fe-ac04-4790-a7c7-174550604e06 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:53.318267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9269b9fe-ac04-4790-a7c7-174550604e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:53.410821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:53.474241 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269b9fe-ac04-4790-a7c7-174550604e06 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:53.610691 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84601cb-74ab-416a-aa45-45644b7ed093 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:53.611297 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84601cb-74ab-416a-aa45-45644b7ed093 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:53.611566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84601cb-74ab-416a-aa45-45644b7ed093 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:53.611854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84601cb-74ab-416a-aa45-45644b7ed093 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:53.613513 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f84601cb-74ab-416a-aa45-45644b7ed093 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:53.668056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:53.713287 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84601cb-74ab-416a-aa45-45644b7ed093 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:53.715633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84601cb-74ab-416a-aa45-45644b7ed093 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:53.734160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 712/1428] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:54.778747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d00fa-2b99-4d29-94c4-dbb012fc250b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:54.779366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d00fa-2b99-4d29-94c4-dbb012fc250b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:54.779366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d00fa-2b99-4d29-94c4-dbb012fc250b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:54.780622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:54.787963 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1b642-4105-488e-8894-3783db941371 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:54.788728 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1b642-4105-488e-8894-3783db941371 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:54.788921 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1b642-4105-488e-8894-3783db941371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:54.870277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-cae1b642-4105-488e-8894-3783db941371 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:54.871515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-cae1b642-4105-488e-8894-3783db941371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:54.983411 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:55.136612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d00fa-2b99-4d29-94c4-dbb012fc250b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:55.138609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f34d00fa-2b99-4d29-94c4-dbb012fc250b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:55.192817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:55.294974 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d00fa-2b99-4d29-94c4-dbb012fc250b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:55.418861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dda3cf-4dfc-41eb-bd5d-b74786421542 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:55.419483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dda3cf-4dfc-41eb-bd5d-b74786421542 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:55.419829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dda3cf-4dfc-41eb-bd5d-b74786421542 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:55.420167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dda3cf-4dfc-41eb-bd5d-b74786421542 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:55.422713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-26dda3cf-4dfc-41eb-bd5d-b74786421542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:55.431092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:55.526042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dda3cf-4dfc-41eb-bd5d-b74786421542 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:55.527872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dda3cf-4dfc-41eb-bd5d-b74786421542 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:55.547274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 718/1431] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:56.595658 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8c055-cbd9-4247-b332-d025c0e2f33f None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:56.596259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8c055-cbd9-4247-b332-d025c0e2f33f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:56.596259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8c055-cbd9-4247-b332-d025c0e2f33f None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:56.598351 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:11:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:56.604939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59381ec-b750-42ca-80da-24712b516c72 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:56.605425 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59381ec-b750-42ca-80da-24712b516c72 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:56.605534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59381ec-b750-42ca-80da-24712b516c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:56.685631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f59381ec-b750-42ca-80da-24712b516c72 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:56.686818 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f59381ec-b750-42ca-80da-24712b516c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:56.763420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:56.795179 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:56.950469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8c055-cbd9-4247-b332-d025c0e2f33f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:56.951918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-12d8c055-cbd9-4247-b332-d025c0e2f33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:56.970174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:57.111019 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8c055-cbd9-4247-b332-d025c0e2f33f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:57.202670 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:57.229714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeea1a6-2a31-4128-a80b-609738670787 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:57.230385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeea1a6-2a31-4128-a80b-609738670787 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:57.230652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeea1a6-2a31-4128-a80b-609738670787 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:57.230887 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeea1a6-2a31-4128-a80b-609738670787 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:57.232484 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0aeea1a6-2a31-4128-a80b-609738670787 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:57.335959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeea1a6-2a31-4128-a80b-609738670787 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:57.337601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeea1a6-2a31-4128-a80b-609738670787 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:57.357690 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 715/1434] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:11:58.441310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f59eb-e9dd-416c-b54d-42cc249c866b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:58.441927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f59eb-e9dd-416c-b54d-42cc249c866b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:58.441927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f59eb-e9dd-416c-b54d-42cc249c866b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:58.443211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:11:58.450119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5e13a-4ebe-484a-bded-66a12cd9707a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:58.450598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5e13a-4ebe-484a-bded-66a12cd9707a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:58.450709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5e13a-4ebe-484a-bded-66a12cd9707a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:58.532175 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-eee5e13a-4ebe-484a-bded-66a12cd9707a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:11:58.533438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-eee5e13a-4ebe-484a-bded-66a12cd9707a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:58.584463 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:58.647310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:11:58.807576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f59eb-e9dd-416c-b54d-42cc249c866b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:11:58.809483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a69f59eb-e9dd-416c-b54d-42cc249c866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:58.986603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f59eb-e9dd-416c-b54d-42cc249c866b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:11:59.042555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:11:59.110856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9adff6f-851c-444c-b348-6e2e4b4d086e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:11:59.111470 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adff6f-851c-444c-b348-6e2e4b4d086e service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:11:59.111748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adff6f-851c-444c-b348-6e2e4b4d086e service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:11:59.112018 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9adff6f-851c-444c-b348-6e2e4b4d086e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:11:59.113685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e9adff6f-851c-444c-b348-6e2e4b4d086e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:11:59.220021 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adff6f-851c-444c-b348-6e2e4b4d086e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:11:59.221654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9adff6f-851c-444c-b348-6e2e4b4d086e service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:11:59.240478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 721/1437] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:11:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:00.326088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0001d6-213c-471d-be1e-59d0eceff8a8 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:00.326717 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0001d6-213c-471d-be1e-59d0eceff8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:00.326717 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0001d6-213c-471d-be1e-59d0eceff8a8 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:00.328256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:12:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:00.338753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e2ed7-f1c1-4429-a67f-15e5d2ff3fb8 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:00.339325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e2ed7-f1c1-4429-a67f-15e5d2ff3fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:00.339325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e2ed7-f1c1-4429-a67f-15e5d2ff3fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:00.430486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5b4e2ed7-f1c1-4429-a67f-15e5d2ff3fb8 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:00.431706 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5b4e2ed7-f1c1-4429-a67f-15e5d2ff3fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:00.433896 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:00.656215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:00.805840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0001d6-213c-471d-be1e-59d0eceff8a8 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:00.807381 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1e0001d6-213c-471d-be1e-59d0eceff8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:00.987742 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0001d6-213c-471d-be1e-59d0eceff8a8 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:01.023934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:01.116020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f150e-c5cf-4626-a95e-7b37a1a8ecf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:01.116639 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f150e-c5cf-4626-a95e-7b37a1a8ecf5 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:01.116925 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f150e-c5cf-4626-a95e-7b37a1a8ecf5 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:01.117189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f150e-c5cf-4626-a95e-7b37a1a8ecf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:01.118788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-794f150e-c5cf-4626-a95e-7b37a1a8ecf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:01.221517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f150e-c5cf-4626-a95e-7b37a1a8ecf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:01.223141 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f150e-c5cf-4626-a95e-7b37a1a8ecf5 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:01.247958 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 718/1440] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:03.470525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd96ef9-7f5a-4f95-8f49-b8584553e441 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:03.471126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd96ef9-7f5a-4f95-8f49-b8584553e441 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:03.471126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd96ef9-7f5a-4f95-8f49-b8584553e441 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:03.472396 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:03.478914 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70473b3c-85b4-4590-8f4b-8d8ef47c072e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:03.479646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70473b3c-85b4-4590-8f4b-8d8ef47c072e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:03.479833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70473b3c-85b4-4590-8f4b-8d8ef47c072e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:03.527882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:03.557493 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-70473b3c-85b4-4590-8f4b-8d8ef47c072e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:03.558796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-70473b3c-85b4-4590-8f4b-8d8ef47c072e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:03.664024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:03.955379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd96ef9-7f5a-4f95-8f49-b8584553e441 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:03.956975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1bd96ef9-7f5a-4f95-8f49-b8584553e441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:04.073175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd96ef9-7f5a-4f95-8f49-b8584553e441 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:04.115501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:04.171519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce4979-dbc8-463e-8f5f-89c4b9fd25c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:04.172123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce4979-dbc8-463e-8f5f-89c4b9fd25c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:04.172400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce4979-dbc8-463e-8f5f-89c4b9fd25c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:04.172704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce4979-dbc8-463e-8f5f-89c4b9fd25c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:04.174322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-75ce4979-dbc8-463e-8f5f-89c4b9fd25c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:04.293990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce4979-dbc8-463e-8f5f-89c4b9fd25c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:04.295315 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce4979-dbc8-463e-8f5f-89c4b9fd25c0 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:04.313865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 724/1443] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:06.632402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903ff71-9d0a-4b3c-9f15-418608c8e7dd None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:06.633047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903ff71-9d0a-4b3c-9f15-418608c8e7dd None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:06.633047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903ff71-9d0a-4b3c-9f15-418608c8e7dd None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:06.634313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:06.639797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1bab6-9b27-4a50-8e9a-e59e676852ed None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:06.640272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1bab6-9b27-4a50-8e9a-e59e676852ed None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:06.640377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1bab6-9b27-4a50-8e9a-e59e676852ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:06.708862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6ff1bab6-9b27-4a50-8e9a-e59e676852ed None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:06.710045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6ff1bab6-9b27-4a50-8e9a-e59e676852ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:06.722907 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:06.812063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:07.100472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903ff71-9d0a-4b3c-9f15-418608c8e7dd None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:07.102769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7903ff71-9d0a-4b3c-9f15-418608c8e7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:07.229270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903ff71-9d0a-4b3c-9f15-418608c8e7dd None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:07.249673 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:07.313826 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924df0b9-29f5-4adf-b5b1-504dbf92a2f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:07.314471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924df0b9-29f5-4adf-b5b1-504dbf92a2f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:07.314875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924df0b9-29f5-4adf-b5b1-504dbf92a2f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:07.315152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924df0b9-29f5-4adf-b5b1-504dbf92a2f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:07.316890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-924df0b9-29f5-4adf-b5b1-504dbf92a2f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:07.444114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924df0b9-29f5-4adf-b5b1-504dbf92a2f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:07.445522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924df0b9-29f5-4adf-b5b1-504dbf92a2f6 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:07.466841 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 721/1446] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:09.929084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee138ed-6f90-4a93-a504-7fa5b178dbe5 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:09.929994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee138ed-6f90-4a93-a504-7fa5b178dbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:09.929994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee138ed-6f90-4a93-a504-7fa5b178dbe5 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:09.932128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:12:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:09.938198 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a59721-ef67-4793-b023-31c55c3beacf None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:09.938701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a59721-ef67-4793-b023-31c55c3beacf None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:09.938701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a59721-ef67-4793-b023-31c55c3beacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:09.986989 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:10.005711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d0a59721-ef67-4793-b023-31c55c3beacf None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:10.006906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d0a59721-ef67-4793-b023-31c55c3beacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:10.113379 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:12:10.402581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee138ed-6f90-4a93-a504-7fa5b178dbe5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:10.405116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1ee138ed-6f90-4a93-a504-7fa5b178dbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:10.524768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee138ed-6f90-4a93-a504-7fa5b178dbe5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:10.565946 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:10.602382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432225e9-b811-4265-9517-06c7304e23d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:10.602992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432225e9-b811-4265-9517-06c7304e23d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:10.603290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432225e9-b811-4265-9517-06c7304e23d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:10.603531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432225e9-b811-4265-9517-06c7304e23d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:10.605670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-432225e9-b811-4265-9517-06c7304e23d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:10.718997 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432225e9-b811-4265-9517-06c7304e23d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:10.720309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432225e9-b811-4265-9517-06c7304e23d0 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:10.738094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 727/1449] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:13.060962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c5876-89f0-492a-b804-cf0d236a36bc None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:13.060962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c5876-89f0-492a-b804-cf0d236a36bc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:13.061869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c5876-89f0-492a-b804-cf0d236a36bc None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:13.063882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:12:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:13.073163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5be903-8cb4-4593-bce2-21f0df2f4b15 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:13.073920 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5be903-8cb4-4593-bce2-21f0df2f4b15 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:13.074033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5be903-8cb4-4593-bce2-21f0df2f4b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:13.142946 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7c5be903-8cb4-4593-bce2-21f0df2f4b15 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:13.144140 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7c5be903-8cb4-4593-bce2-21f0df2f4b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:13.164718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:13.243454 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:12:13.530863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c5876-89f0-492a-b804-cf0d236a36bc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:13.532537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3d2c5876-89f0-492a-b804-cf0d236a36bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:13.649140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c5876-89f0-492a-b804-cf0d236a36bc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:13.670959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:13.727214 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180b460-89d7-4c27-9b07-750df4f6a436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:13.727815 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180b460-89d7-4c27-9b07-750df4f6a436 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:13.728087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180b460-89d7-4c27-9b07-750df4f6a436 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:13.728366 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180b460-89d7-4c27-9b07-750df4f6a436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:13.730158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6180b460-89d7-4c27-9b07-750df4f6a436 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:13.842964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180b460-89d7-4c27-9b07-750df4f6a436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:13.844420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180b460-89d7-4c27-9b07-750df4f6a436 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:13.862891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 724/1452] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:16.052517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5958c-141d-4beb-8d9d-ab6eb0395f46 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:16.053479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5958c-141d-4beb-8d9d-ab6eb0395f46 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:16.053479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5958c-141d-4beb-8d9d-ab6eb0395f46 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:16.055613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:12:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:16.062457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48076610-38e5-46aa-b879-f2e060a331d7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:16.062894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48076610-38e5-46aa-b879-f2e060a331d7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:16.063016 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48076610-38e5-46aa-b879-f2e060a331d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:16.146768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:16.150108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-48076610-38e5-46aa-b879-f2e060a331d7 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:16.152134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-48076610-38e5-46aa-b879-f2e060a331d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:16.253978 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:16.544496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5958c-141d-4beb-8d9d-ab6eb0395f46 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:16.546227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-26a5958c-141d-4beb-8d9d-ab6eb0395f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:16.652166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5958c-141d-4beb-8d9d-ab6eb0395f46 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:16.703161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:16.727650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3eaab3-d11b-43fb-9bff-62431cda521e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:16.728270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3eaab3-d11b-43fb-9bff-62431cda521e service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:16.728516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3eaab3-d11b-43fb-9bff-62431cda521e service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:16.728845 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3eaab3-d11b-43fb-9bff-62431cda521e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:16.730411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8b3eaab3-d11b-43fb-9bff-62431cda521e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:16.854707 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3eaab3-d11b-43fb-9bff-62431cda521e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:16.856125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3eaab3-d11b-43fb-9bff-62431cda521e service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:16.875255 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 730/1455] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:19.124822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbc92f-e7b1-4286-b059-64f90b0c03f9 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:19.125707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbc92f-e7b1-4286-b059-64f90b0c03f9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:19.125707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbc92f-e7b1-4286-b059-64f90b0c03f9 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:19.127850 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:12:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:19.134264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e0889-01d2-4cc0-b388-8d3a39560ee6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:19.135009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e0889-01d2-4cc0-b388-8d3a39560ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:19.135508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e0889-01d2-4cc0-b388-8d3a39560ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:19.221614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-986e0889-01d2-4cc0-b388-8d3a39560ee6 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:19.222823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-986e0889-01d2-4cc0-b388-8d3a39560ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:19.254177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:19.334341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:19.636622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbc92f-e7b1-4286-b059-64f90b0c03f9 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:19.638205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-11cbc92f-e7b1-4286-b059-64f90b0c03f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:19.748331 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbc92f-e7b1-4286-b059-64f90b0c03f9 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:19.782443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:19.830064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bae986-0cf4-4efc-bddd-584382e9fc15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:19.830664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bae986-0cf4-4efc-bddd-584382e9fc15 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:19.830925 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bae986-0cf4-4efc-bddd-584382e9fc15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:19.831187 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bae986-0cf4-4efc-bddd-584382e9fc15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:19.832814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-85bae986-0cf4-4efc-bddd-584382e9fc15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:19.945480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bae986-0cf4-4efc-bddd-584382e9fc15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:19.946900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bae986-0cf4-4efc-bddd-584382e9fc15 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:19.965880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 727/1458] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:22.186079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d94187-3f40-4d54-a27c-386db5efbc17 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:22.186712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d94187-3f40-4d54-a27c-386db5efbc17 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:22.186712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d94187-3f40-4d54-a27c-386db5efbc17 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:22.187940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:22.193672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce302f49-43ba-43e4-89c7-bdc9dbd0fd32 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:22.194094 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce302f49-43ba-43e4-89c7-bdc9dbd0fd32 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:22.194219 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce302f49-43ba-43e4-89c7-bdc9dbd0fd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:22.263944 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ce302f49-43ba-43e4-89c7-bdc9dbd0fd32 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:22.265179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ce302f49-43ba-43e4-89c7-bdc9dbd0fd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:22.275459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:22.371390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:22.653544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d94187-3f40-4d54-a27c-386db5efbc17 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:22.655124 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-18d94187-3f40-4d54-a27c-386db5efbc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:22.764099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d94187-3f40-4d54-a27c-386db5efbc17 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:22.821260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:22.839825 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b53aab-4fed-440a-844e-c53272c4c703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:22.840418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53aab-4fed-440a-844e-c53272c4c703 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:22.840716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53aab-4fed-440a-844e-c53272c4c703 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:22.840981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53aab-4fed-440a-844e-c53272c4c703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:22.842572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e2b53aab-4fed-440a-844e-c53272c4c703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:22.958096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b53aab-4fed-440a-844e-c53272c4c703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:22.959526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b53aab-4fed-440a-844e-c53272c4c703 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:22.978130 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 733/1461] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:12:25.170173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919289b-c3f4-4aac-b291-402b9f1610f6 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:25.170773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919289b-c3f4-4aac-b291-402b9f1610f6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:25.170773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919289b-c3f4-4aac-b291-402b9f1610f6 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:25.173098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:12:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 15:12:25.182623 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b64a-37af-4154-a619-8fdaaf08810a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:25.183088 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b64a-37af-4154-a619-8fdaaf08810a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:25.183440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b64a-37af-4154-a619-8fdaaf08810a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:25.252170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5138b64a-37af-4154-a619-8fdaaf08810a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:25.253421 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5138b64a-37af-4154-a619-8fdaaf08810a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:25.293019 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:25.356569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:25.636264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919289b-c3f4-4aac-b291-402b9f1610f6 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:25.639145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a919289b-c3f4-4aac-b291-402b9f1610f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:25.758961 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919289b-c3f4-4aac-b291-402b9f1610f6 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:25.804177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:25.845012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf156b4-de12-4e70-afbc-6bdae222b074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:25.845611 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf156b4-de12-4e70-afbc-6bdae222b074 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:25.845887 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf156b4-de12-4e70-afbc-6bdae222b074 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:25.846151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf156b4-de12-4e70-afbc-6bdae222b074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:25.847755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7bf156b4-de12-4e70-afbc-6bdae222b074 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:25.961219 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf156b4-de12-4e70-afbc-6bdae222b074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:25.962579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf156b4-de12-4e70-afbc-6bdae222b074 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:25.982272 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 730/1464] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:28.147643 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6cf34-0141-40fd-9660-ebe1944f7a2a None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:28.148257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6cf34-0141-40fd-9660-ebe1944f7a2a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:28.148257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6cf34-0141-40fd-9660-ebe1944f7a2a None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:28.149587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:28.156077 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a42d37-3bb7-4f0a-8d05-91cc8c5f7b78 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:28.156863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a42d37-3bb7-4f0a-8d05-91cc8c5f7b78 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:28.157034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a42d37-3bb7-4f0a-8d05-91cc8c5f7b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:28.230519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-29a42d37-3bb7-4f0a-8d05-91cc8c5f7b78 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:28.231738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-29a42d37-3bb7-4f0a-8d05-91cc8c5f7b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:28.258028 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:28.340952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:28.623322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6cf34-0141-40fd-9660-ebe1944f7a2a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:28.625903 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d5b6cf34-0141-40fd-9660-ebe1944f7a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:28.765047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6cf34-0141-40fd-9660-ebe1944f7a2a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:28.835944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:28.843412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b20841-132c-46d5-85d7-afa8e7fa245f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:28.844018 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b20841-132c-46d5-85d7-afa8e7fa245f service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:28.844293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b20841-132c-46d5-85d7-afa8e7fa245f service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:28.844557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b20841-132c-46d5-85d7-afa8e7fa245f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:28.846186 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-09b20841-132c-46d5-85d7-afa8e7fa245f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:28.966341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b20841-132c-46d5-85d7-afa8e7fa245f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:28.968419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b20841-132c-46d5-85d7-afa8e7fa245f service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:28.989201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 736/1467] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:31.157128 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c9f56-1062-4281-a4fe-24d568352e13 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:31.157701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c9f56-1062-4281-a4fe-24d568352e13 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:31.157701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c9f56-1062-4281-a4fe-24d568352e13 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:31.159784 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:12:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:31.169571 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde6ad-e2a0-4831-838d-9a2c75b19dac None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:31.170349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde6ad-e2a0-4831-838d-9a2c75b19dac None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:31.170349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dde6ad-e2a0-4831-838d-9a2c75b19dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:31.253252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-05dde6ad-e2a0-4831-838d-9a2c75b19dac None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:31.254454 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-05dde6ad-e2a0-4831-838d-9a2c75b19dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:31.309778 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:31.360397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:31.643435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c9f56-1062-4281-a4fe-24d568352e13 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:31.645971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ad6c9f56-1062-4281-a4fe-24d568352e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:31.759247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c9f56-1062-4281-a4fe-24d568352e13 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:31.809103 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:31.836150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f29b0-55fa-49d4-ab3e-1530c9a58d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:31.836786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f29b0-55fa-49d4-ab3e-1530c9a58d2a service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:31.837053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f29b0-55fa-49d4-ab3e-1530c9a58d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:31.837310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f29b0-55fa-49d4-ab3e-1530c9a58d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:31.839439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b51f29b0-55fa-49d4-ab3e-1530c9a58d2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:31.965338 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f29b0-55fa-49d4-ab3e-1530c9a58d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:31.966702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f29b0-55fa-49d4-ab3e-1530c9a58d2a service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:31.985134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 733/1470] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:41.358782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=22 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:41.982702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=66 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:52.213319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0b8a6-473a-4877-8a33-66473b85ecb9 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:52.214310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0b8a6-473a-4877-8a33-66473b85ecb9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:52.214310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0b8a6-473a-4877-8a33-66473b85ecb9 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:52.216397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 738/1471] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:12:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:52.221820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cb1a9-6b16-4b19-9ea0-7396b1e55ff6 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:52.222309 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cb1a9-6b16-4b19-9ea0-7396b1e55ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:52.222309 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8cb1a9-6b16-4b19-9ea0-7396b1e55ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:52.289370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0f8cb1a9-6b16-4b19-9ea0-7396b1e55ff6 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:52.290583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0f8cb1a9-6b16-4b19-9ea0-7396b1e55ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:52.392996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 734/1472] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:52.708955 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcf770-2239-4ead-bb81-9833c395c3ae None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:52.709236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcf770-2239-4ead-bb81-9833c395c3ae None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:52.709503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bcf770-2239-4ead-bb81-9833c395c3ae None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:52.710881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 739/1473] 10.222.0.75 () {54 vars in 962 bytes} [Fri Sep 23 15:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:52.717157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e841642-bbb7-42cf-960e-35c60a0838f9 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:52.717487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e841642-bbb7-42cf-960e-35c60a0838f9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:52.717865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e841642-bbb7-42cf-960e-35c60a0838f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:52.798724 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7e841642-bbb7-42cf-960e-35c60a0838f9 None None] MFA Rules not processed for user `ac783fbe95ea4b74bd46af415c9734fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:52.799846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7e841642-bbb7-42cf-960e-35c60a0838f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:52.825099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:52.884449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 735/1474] 10.222.0.75 () {58 vars in 1059 bytes} [Fri Sep 23 15:12:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:52.892891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcf770-2239-4ead-bb81-9833c395c3ae None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:52.894463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-34bcf770-2239-4ead-bb81-9833c395c3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:53.007320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bcf770-2239-4ead-bb81-9833c395c3ae None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:53.087175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701660d4-9ca1-4444-86ec-fd25f5225229 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac783fbe95ea4b74bd46af415c9734fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:53.087785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701660d4-9ca1-4444-86ec-fd25f5225229 service glance] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:53.088060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701660d4-9ca1-4444-86ec-fd25f5225229 service glance] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:53.088341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701660d4-9ca1-4444-86ec-fd25f5225229 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:53.090053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-701660d4-9ca1-4444-86ec-fd25f5225229 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:53.151364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:53.207905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701660d4-9ca1-4444-86ec-fd25f5225229 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:53.209349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701660d4-9ca1-4444-86ec-fd25f5225229 service glance] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:53.228215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 740/1475] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:12:55.476808 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d90ca-3817-41b7-8135-e39e06673689 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:55.477380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d90ca-3817-41b7-8135-e39e06673689 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:55.477380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d90ca-3817-41b7-8135-e39e06673689 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:55.478757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 736/1476] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:55.484345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11323891-1b99-4c4a-a809-825ea754b8a4 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:55.484807 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11323891-1b99-4c4a-a809-825ea754b8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:55.484928 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11323891-1b99-4c4a-a809-825ea754b8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:55.562276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-11323891-1b99-4c4a-a809-825ea754b8a4 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:55.563528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-11323891-1b99-4c4a-a809-825ea754b8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:55.667950 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 741/1477] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:55.956210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d90ca-3817-41b7-8135-e39e06673689 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:55.957785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fc5d90ca-3817-41b7-8135-e39e06673689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:56.102638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d90ca-3817-41b7-8135-e39e06673689 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:56.162305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:56.217042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb604be-c558-4be9-a7fa-f7da6722f23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:56.217566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb604be-c558-4be9-a7fa-f7da6722f23b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:56.217832 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb604be-c558-4be9-a7fa-f7da6722f23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:56.218098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb604be-c558-4be9-a7fa-f7da6722f23b admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:56.218739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb604be-c558-4be9-a7fa-f7da6722f23b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:56.220546 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-4fb604be-c558-4be9-a7fa-f7da6722f23b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:12:56.220546 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Sep 23 15:12:56.220546 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: Sep 23 15:12:56.221723 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb604be-c558-4be9-a7fa-f7da6722f23b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:56.242708 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 737/1478] 10.222.0.75 () {56 vars in 1108 bytes} [Fri Sep 23 15:12:55 2022] GET /identity/v3/projects => generated 2553 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:12:58.413929 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c1bd4-7373-4cd2-aa39-3622b5739505 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:58.414558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c1bd4-7373-4cd2-aa39-3622b5739505 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:58.414558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c1bd4-7373-4cd2-aa39-3622b5739505 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:58.416458 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 742/1479] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:12:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:12:58.424957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef38bf-4ad7-4041-9481-6a6a0a452326 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:58.425389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef38bf-4ad7-4041-9481-6a6a0a452326 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:58.425511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef38bf-4ad7-4041-9481-6a6a0a452326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:58.497809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-cfef38bf-4ad7-4041-9481-6a6a0a452326 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:12:58.499059 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-cfef38bf-4ad7-4041-9481-6a6a0a452326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:58.600969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 738/1480] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:12:58.903828 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c1bd4-7373-4cd2-aa39-3622b5739505 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:12:58.905464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4e0c1bd4-7373-4cd2-aa39-3622b5739505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:58.916131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:59.056904 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c1bd4-7373-4cd2-aa39-3622b5739505 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:12:59.141546 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:12:59.175256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf77f1-9d75-4f95-bdd1-21bb9387a0a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:12:59.175926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf77f1-9d75-4f95-bdd1-21bb9387a0a7 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:12:59.176232 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf77f1-9d75-4f95-bdd1-21bb9387a0a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:12:59.176553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf77f1-9d75-4f95-bdd1-21bb9387a0a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:12:59.178201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5eaf77f1-9d75-4f95-bdd1-21bb9387a0a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:12:59.281271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf77f1-9d75-4f95-bdd1-21bb9387a0a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:12:59.282860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf77f1-9d75-4f95-bdd1-21bb9387a0a7 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:12:59.301377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 743/1481] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:13:01.718916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854a46a-d7bc-4481-ab37-147eb0d8b5fc None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:01.719507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854a46a-d7bc-4481-ab37-147eb0d8b5fc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:01.719507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854a46a-d7bc-4481-ab37-147eb0d8b5fc None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:01.720843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 739/1482] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:01.727538 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2c57-7e73-4f82-9961-d5bbef04c55c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:01.728018 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2c57-7e73-4f82-9961-d5bbef04c55c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:01.728112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb2c57-7e73-4f82-9961-d5bbef04c55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:01.824797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4dfb2c57-7e73-4f82-9961-d5bbef04c55c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:01.826125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4dfb2c57-7e73-4f82-9961-d5bbef04c55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:01.892088 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:01.950286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 744/1483] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:02.252750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854a46a-d7bc-4481-ab37-147eb0d8b5fc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:02.254341 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c854a46a-d7bc-4481-ab37-147eb0d8b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:02.280225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:02.381410 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854a46a-d7bc-4481-ab37-147eb0d8b5fc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:02.458579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb76ec8-9191-4c9c-8686-6a65b60ccd75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:02.459093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb76ec8-9191-4c9c-8686-6a65b60ccd75 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:02.459342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb76ec8-9191-4c9c-8686-6a65b60ccd75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:02.459621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb76ec8-9191-4c9c-8686-6a65b60ccd75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:02.461276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3cb76ec8-9191-4c9c-8686-6a65b60ccd75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:02.539624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:02.581211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb76ec8-9191-4c9c-8686-6a65b60ccd75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:02.582672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb76ec8-9191-4c9c-8686-6a65b60ccd75 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:02.602282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 740/1484] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:04.835482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b0394-f19b-4aea-a185-9c7487251837 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:04.836420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b0394-f19b-4aea-a185-9c7487251837 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:04.836420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b0394-f19b-4aea-a185-9c7487251837 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:04.838562 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 745/1485] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:13:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:04.847162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76de1cb-e26f-44fd-a13d-15de83f4957c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:04.847611 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76de1cb-e26f-44fd-a13d-15de83f4957c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:04.847711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76de1cb-e26f-44fd-a13d-15de83f4957c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:04.914705 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e76de1cb-e26f-44fd-a13d-15de83f4957c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:04.915897 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e76de1cb-e26f-44fd-a13d-15de83f4957c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:05.026471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 741/1486] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:13:05.315682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b0394-f19b-4aea-a185-9c7487251837 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:05.317286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ea8b0394-f19b-4aea-a185-9c7487251837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:05.429260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b0394-f19b-4aea-a185-9c7487251837 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:05.510461 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:05.512290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666f52a-bfbc-484c-a2c7-83a67e0d0dd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:05.513075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666f52a-bfbc-484c-a2c7-83a67e0d0dd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:05.513384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666f52a-bfbc-484c-a2c7-83a67e0d0dd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:05.513701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666f52a-bfbc-484c-a2c7-83a67e0d0dd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:05.516006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a666f52a-bfbc-484c-a2c7-83a67e0d0dd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:05.639406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666f52a-bfbc-484c-a2c7-83a67e0d0dd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:05.640875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666f52a-bfbc-484c-a2c7-83a67e0d0dd0 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:05.660281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 746/1487] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:08.022538 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d7ca9-728d-420e-91d0-51a0aae10d4c None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:08.023464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d7ca9-728d-420e-91d0-51a0aae10d4c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:08.023464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d7ca9-728d-420e-91d0-51a0aae10d4c None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:08.025544 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 742/1488] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:13:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:08.031877 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413613bd-aa99-4109-9a3d-26f05bf8567c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:08.032330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413613bd-aa99-4109-9a3d-26f05bf8567c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:08.032423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413613bd-aa99-4109-9a3d-26f05bf8567c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:08.102428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-413613bd-aa99-4109-9a3d-26f05bf8567c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:08.103629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-413613bd-aa99-4109-9a3d-26f05bf8567c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:08.161075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:08.212144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 747/1489] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:13:08.506261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d7ca9-728d-420e-91d0-51a0aae10d4c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:08.507836 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ac7d7ca9-728d-420e-91d0-51a0aae10d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:08.547732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:08.621509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d7ca9-728d-420e-91d0-51a0aae10d4c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:08.700900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b872dd-a9c7-4c53-9218-0ba953c53b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:08.701474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b872dd-a9c7-4c53-9218-0ba953c53b83 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:08.701779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b872dd-a9c7-4c53-9218-0ba953c53b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:08.702039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b872dd-a9c7-4c53-9218-0ba953c53b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:08.703630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f5b872dd-a9c7-4c53-9218-0ba953c53b83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:08.788362 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:08.818277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b872dd-a9c7-4c53-9218-0ba953c53b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:08.819722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b872dd-a9c7-4c53-9218-0ba953c53b83 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:08.838860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 743/1490] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:11.235696 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84339ed-3b8a-4af3-b127-4c1c53acd6d9 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:11.236306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84339ed-3b8a-4af3-b127-4c1c53acd6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:11.236306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84339ed-3b8a-4af3-b127-4c1c53acd6d9 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:11.237963 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 748/1491] 10.222.0.75 () {54 vars in 927 bytes} [Fri Sep 23 15:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:13:11.244358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b6801-719a-4775-a680-e89c4c28ee9e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:11.245137 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b6801-719a-4775-a680-e89c4c28ee9e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:11.245323 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b6801-719a-4775-a680-e89c4c28ee9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:11.327038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1d2b6801-719a-4775-a680-e89c4c28ee9e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:13:11.328236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1d2b6801-719a-4775-a680-e89c4c28ee9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:11.432631 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 744/1492] 10.222.0.75 () {58 vars in 1024 bytes} [Fri Sep 23 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:13:11.722252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84339ed-3b8a-4af3-b127-4c1c53acd6d9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:13:11.724745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e84339ed-3b8a-4af3-b127-4c1c53acd6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:11.835815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84339ed-3b8a-4af3-b127-4c1c53acd6d9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:13:11.914827 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e996ca-ac96-4d83-879f-812ad6208728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:13:11.915430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e996ca-ac96-4d83-879f-812ad6208728 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:13:11.915701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e996ca-ac96-4d83-879f-812ad6208728 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:13:11.915970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e996ca-ac96-4d83-879f-812ad6208728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:13:11.917846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a0e996ca-ac96-4d83-879f-812ad6208728 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:13:11.926877 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:12.035336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e996ca-ac96-4d83-879f-812ad6208728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:13:12.036692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e996ca-ac96-4d83-879f-812ad6208728 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:13:12.055053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 749/1493] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:13:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:13:21.430969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=82 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:13:22.052633 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=50 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:25.729038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecf93b-d606-4f42-83ff-74441cf96b4b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:25.729659 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecf93b-d606-4f42-83ff-74441cf96b4b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:25.729659 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecf93b-d606-4f42-83ff-74441cf96b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:25.825580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b6ecf93b-d606-4f42-83ff-74441cf96b4b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:25.826831 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b6ecf93b-d606-4f42-83ff-74441cf96b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:25.944575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 745/1494] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:14:25.953224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d2079-599a-4765-b9da-58823dbac0e5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:25.953751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d2079-599a-4765-b9da-58823dbac0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:25.953751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d2079-599a-4765-b9da-58823dbac0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:26.037912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e08d2079-599a-4765-b9da-58823dbac0e5 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:26.039127 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e08d2079-599a-4765-b9da-58823dbac0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:26.145976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 750/1495] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:14:26.151821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecf93b-d606-4f42-83ff-74441cf96b4b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:26.153083 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b6ecf93b-d606-4f42-83ff-74441cf96b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:26.165202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecf93b-d606-4f42-83ff-74441cf96b4b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:26.229934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:26.264343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d414bde-4205-4f6b-a9bd-86da83334c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:26.264983 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d414bde-4205-4f6b-a9bd-86da83334c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:26.265254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d414bde-4205-4f6b-a9bd-86da83334c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:26.265520 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d414bde-4205-4f6b-a9bd-86da83334c58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:26.266164 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d414bde-4205-4f6b-a9bd-86da83334c58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:14:26.266465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d414bde-4205-4f6b-a9bd-86da83334c58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:26.267469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-0d414bde-4205-4f6b-a9bd-86da83334c58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:14:26.267469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Sep 23 15:14:26.267469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: Sep 23 15:14:26.268372 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d414bde-4205-4f6b-a9bd-86da83334c58 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:26.279971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 746/1496] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:14:26.288597 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d2079-599a-4765-b9da-58823dbac0e5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:26.298450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d2079-599a-4765-b9da-58823dbac0e5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:26.369122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:26.404791 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d697ad5-2763-4b8a-8b04-9731ab57b1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:26.405333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d697ad5-2763-4b8a-8b04-9731ab57b1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:26.405602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d697ad5-2763-4b8a-8b04-9731ab57b1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:26.405877 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d697ad5-2763-4b8a-8b04-9731ab57b1d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:26.406530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d697ad5-2763-4b8a-8b04-9731ab57b1d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:26.408314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-3d697ad5-2763-4b8a-8b04-9731ab57b1d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:14:26.408314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Sep 23 15:14:26.408314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: Sep 23 15:14:26.409561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d697ad5-2763-4b8a-8b04-9731ab57b1d2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:26.473141 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 751/1497] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:14:26 2022] POST /identity/v3/projects => generated 379 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:14:26.477797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d414bde-4205-4f6b-a9bd-86da83334c58 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:26.479340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0d414bde-4205-4f6b-a9bd-86da83334c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:26.512915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d414bde-4205-4f6b-a9bd-86da83334c58 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:26.614780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d66fe-ae0d-4453-9004-5f73bef13034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:26.615393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d66fe-ae0d-4453-9004-5f73bef13034 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:26.615661 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d66fe-ae0d-4453-9004-5f73bef13034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:26.615931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d66fe-ae0d-4453-9004-5f73bef13034 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:26.616562 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d66fe-ae0d-4453-9004-5f73bef13034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:26.617971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-4b6d66fe-ae0d-4453-9004-5f73bef13034 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:14:26.617971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Sep 23 15:14:26.617971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: Sep 23 15:14:26.618912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d66fe-ae0d-4453-9004-5f73bef13034 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:26.665473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 747/1498] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:14:26 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:14:26.670084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d697ad5-2763-4b8a-8b04-9731ab57b1d2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:26.671609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3d697ad5-2763-4b8a-8b04-9731ab57b1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:26.735916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d697ad5-2763-4b8a-8b04-9731ab57b1d2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:26.822770 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:26.833767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc51f1-fb7e-4eb3-8c13-9f962f13970e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:26.834375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc51f1-fb7e-4eb3-8c13-9f962f13970e admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:26.834669 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc51f1-fb7e-4eb3-8c13-9f962f13970e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:26.834928 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc51f1-fb7e-4eb3-8c13-9f962f13970e admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:26.835580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc51f1-fb7e-4eb3-8c13-9f962f13970e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:26.836680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-f7fc51f1-fb7e-4eb3-8c13-9f962f13970e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:14:26.836680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Sep 23 15:14:26.836680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: Sep 23 15:14:26.837561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc51f1-fb7e-4eb3-8c13-9f962f13970e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:26.850539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 752/1499] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:14:26.855150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d66fe-ae0d-4453-9004-5f73bef13034 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:26.856768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4b6d66fe-ae0d-4453-9004-5f73bef13034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:26.869258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d66fe-ae0d-4453-9004-5f73bef13034 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:26.895740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:26.972232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64a920-7c5c-498d-8e2c-233d3660ff10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:26.972792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64a920-7c5c-498d-8e2c-233d3660ff10 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:26.973062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64a920-7c5c-498d-8e2c-233d3660ff10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:26.973329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64a920-7c5c-498d-8e2c-233d3660ff10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:26.973982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64a920-7c5c-498d-8e2c-233d3660ff10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:26.975007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-8b64a920-7c5c-498d-8e2c-233d3660ff10 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:14:26.975007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Sep 23 15:14:26.975007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: Sep 23 15:14:26.975892 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64a920-7c5c-498d-8e2c-233d3660ff10 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:26.989816 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 748/1500] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:14:26.994389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc51f1-fb7e-4eb3-8c13-9f962f13970e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:26.995921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f7fc51f1-fb7e-4eb3-8c13-9f962f13970e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:27.032610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc51f1-fb7e-4eb3-8c13-9f962f13970e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:27.129753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73960c0-ecd5-45e3-a011-7a9e7bfeefae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:27.130438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73960c0-ecd5-45e3-a011-7a9e7bfeefae admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:27.130684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73960c0-ecd5-45e3-a011-7a9e7bfeefae admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:27.130985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73960c0-ecd5-45e3-a011-7a9e7bfeefae admin admin] PATH_INFO: `/v3/projects/71a8aa3067f941418a0f15004fe29a53/users/5f0d348b5af440f382d3295f69d92dc2/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:27.160456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73960c0-ecd5-45e3-a011-7a9e7bfeefae admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a8aa3067f941418a0f15004fe29a53, user_id=5f0d348b5af440f382d3295f69d92dc2, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:27.162067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-e73960c0-ecd5-45e3-a011-7a9e7bfeefae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:14:27.162067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Sep 23 15:14:27.162067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: Sep 23 15:14:27.164248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73960c0-ecd5-45e3-a011-7a9e7bfeefae admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:27.192619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 753/1501] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:14:26 2022] PUT /identity/v3/projects/71a8aa3067f941418a0f15004fe29a53/users/5f0d348b5af440f382d3295f69d92dc2/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:14:27.205585 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa147f-4b26-4399-8893-4f826615c5c2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:27.205865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa147f-4b26-4399-8893-4f826615c5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:27.206121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa147f-4b26-4399-8893-4f826615c5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:27.272705 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-61aa147f-4b26-4399-8893-4f826615c5c2 None None] MFA Rules not processed for user `5f0d348b5af440f382d3295f69d92dc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:27.273823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-61aa147f-4b26-4399-8893-4f826615c5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:27.366822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 749/1502] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:14:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:14:27.380614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46755d8c-717b-4a51-ad4a-4f2bb243ea1c None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:27.380916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46755d8c-717b-4a51-ad4a-4f2bb243ea1c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:27.381186 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46755d8c-717b-4a51-ad4a-4f2bb243ea1c None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:27.382516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 754/1503] 10.222.0.75 () {54 vars in 963 bytes} [Fri Sep 23 15:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:14:27.387789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffa8b4-c2cb-4137-ad7b-e3589aba1249 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:27.388069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffa8b4-c2cb-4137-ad7b-e3589aba1249 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:27.388336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffa8b4-c2cb-4137-ad7b-e3589aba1249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:27.422301 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:27.483826 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e0ffa8b4-c2cb-4137-ad7b-e3589aba1249 None None] MFA Rules not processed for user `1cd51776db3d426ea1bf1133344fa187`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:27.485147 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e0ffa8b4-c2cb-4137-ad7b-e3589aba1249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:27.567621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 750/1504] 10.222.0.75 () {58 vars in 1060 bytes} [Fri Sep 23 15:14:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:14:27.576139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46755d8c-717b-4a51-ad4a-4f2bb243ea1c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:27.578380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-46755d8c-717b-4a51-ad4a-4f2bb243ea1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:27.634974 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:27.694743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46755d8c-717b-4a51-ad4a-4f2bb243ea1c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:27.778003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d5f96-6ed9-4417-a6e4-b0d1c793eb86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:27.778561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d5f96-6ed9-4417-a6e4-b0d1c793eb86 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:27.778837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d5f96-6ed9-4417-a6e4-b0d1c793eb86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:27.779121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d5f96-6ed9-4417-a6e4-b0d1c793eb86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:27.780783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-461d5f96-6ed9-4417-a6e4-b0d1c793eb86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:27.888899 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d5f96-6ed9-4417-a6e4-b0d1c793eb86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:27.890331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d5f96-6ed9-4417-a6e4-b0d1c793eb86 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:27.893779 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:27.909906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 755/1505] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:14:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:14:27.957629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c41f2-9b1e-45db-b390-58d13e507474 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:27.958080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c41f2-9b1e-45db-b390-58d13e507474 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:27.958197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c41f2-9b1e-45db-b390-58d13e507474 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:27.959407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 751/1506] 10.222.0.75 () {52 vars in 830 bytes} [Fri Sep 23 15:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:14:27.963124 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d5f96-6ed9-4417-a6e4-b0d1c793eb86 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:27.964348 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-461d5f96-6ed9-4417-a6e4-b0d1c793eb86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:28.069542 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d5f96-6ed9-4417-a6e4-b0d1c793eb86 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:28.169270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:28.169897 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:28.170164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:28.170429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] PATH_INFO: `/v3/users/5f0d348b5af440f382d3295f69d92dc2` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:28.183890 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f0d348b5af440f382d3295f69d92dc2)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:28.184927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:14:28.184927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Sep 23 15:14:28.184927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: Sep 23 15:14:28.186141 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:28.196137 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:28.258901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f0d348b5af440f382d3295f69d92dc2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:14:28.273572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f0d348b5af440f382d3295f69d92dc2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:14:28.300357 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f0d348b5af440f382d3295f69d92dc2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:14:28.312359 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f0d348b5af440f382d3295f69d92dc2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:14:28.312893 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f0d348b5af440f382d3295f69d92dc2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:14:28.318474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 756/1507] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:14:27 2022] DELETE /identity/v3/users/5f0d348b5af440f382d3295f69d92dc2 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:14:28.323272 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003c7ed-af08-4699-a825-481439f0c7af None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:28.323494 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003c7ed-af08-4699-a825-481439f0c7af None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:28.323763 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003c7ed-af08-4699-a825-481439f0c7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:28.384439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-8003c7ed-af08-4699-a825-481439f0c7af None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:14:28.385667 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8003c7ed-af08-4699-a825-481439f0c7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:28.467824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:28.607607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 752/1508] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:14:28.621098 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:28.622369 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:28.741002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ca80f-bbb4-430f-9a62-f9dc65f05415 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:28.805902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:28.834183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fba74-a0a2-4489-afaf-9ecb6a0463cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:28.834790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fba74-a0a2-4489-afaf-9ecb6a0463cc service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:28.835062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fba74-a0a2-4489-afaf-9ecb6a0463cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:28.835332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fba74-a0a2-4489-afaf-9ecb6a0463cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:28.837521 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5e1fba74-a0a2-4489-afaf-9ecb6a0463cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:28.956265 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fba74-a0a2-4489-afaf-9ecb6a0463cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:28.958228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fba74-a0a2-4489-afaf-9ecb6a0463cc service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:28.980153 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 757/1509] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:14:29.318376 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003c7ed-af08-4699-a825-481439f0c7af None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:14:29.319637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8003c7ed-af08-4699-a825-481439f0c7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:14:29.331497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003c7ed-af08-4699-a825-481439f0c7af None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:14:29.433710 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:14:29.434333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:14:29.434603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:14:29.434865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc admin admin] PATH_INFO: `/v3/projects/71a8aa3067f941418a0f15004fe29a53` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:14:29.445934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a8aa3067f941418a0f15004fe29a53)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:14:29.446961 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:14:29.446961 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Sep 23 15:14:29.446961 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: Sep 23 15:14:29.448134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:14:29.509400 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a8aa3067f941418a0f15004fe29a53', 'request_id': 'req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:14:29.520327 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a8aa3067f941418a0f15004fe29a53', 'request_id': 'req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:14:29.533995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a8aa3067f941418a0f15004fe29a53', 'request_id': 'req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:14:29.558797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc admin admin] The token cache is being invalidate because project 71a8aa3067f941418a0f15004fe29a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:14:29.559078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:14:29.559602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8f0357ff-e9e5-4a88-974a-c5a7a0fb48fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:14:29.560971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 753/1510] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:14:29 2022] DELETE /identity/v3/projects/71a8aa3067f941418a0f15004fe29a53 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:14:38.978009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=62 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:14:39.556449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:15:57.555138 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86503c-232d-4ada-baba-fc0a2c470a3d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:15:57.556198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86503c-232d-4ada-baba-fc0a2c470a3d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:15:57.556473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86503c-232d-4ada-baba-fc0a2c470a3d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:15:57.557923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:15:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:15:57.563143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d120dad-01e1-4a28-afd6-33bad667f0bc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:15:57.563823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d120dad-01e1-4a28-afd6-33bad667f0bc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:15:57.563823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d120dad-01e1-4a28-afd6-33bad667f0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:15:57.651346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3d120dad-01e1-4a28-afd6-33bad667f0bc None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:15:57.652545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3d120dad-01e1-4a28-afd6-33bad667f0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:15:57.765643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:15:57.923189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86503c-232d-4ada-baba-fc0a2c470a3d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:15:57.925670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-db86503c-232d-4ada-baba-fc0a2c470a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:15:58.076211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86503c-232d-4ada-baba-fc0a2c470a3d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:15:58.084156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1412c38-3387-4ba6-8ca5-60f69814759a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:15:58.084817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1412c38-3387-4ba6-8ca5-60f69814759a service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:15:58.085089 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1412c38-3387-4ba6-8ca5-60f69814759a service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:15:58.085354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1412c38-3387-4ba6-8ca5-60f69814759a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:15:58.095369 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1412c38-3387-4ba6-8ca5-60f69814759a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:15:58.097360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1412c38-3387-4ba6-8ca5-60f69814759a service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:15:58.107596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 759/1513] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:15:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:15:59.212078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c7cd4-cabf-4b29-bfef-a145cae99155 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:15:59.212535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c7cd4-cabf-4b29-bfef-a145cae99155 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:15:59.212669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c7cd4-cabf-4b29-bfef-a145cae99155 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:15:59.213948 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:15:59.221276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c3c3f-ad33-4644-9c82-4da4ceaf932f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:15:59.221774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c3c3f-ad33-4644-9c82-4da4ceaf932f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:15:59.221774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c3c3f-ad33-4644-9c82-4da4ceaf932f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:15:59.247526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f00c3c3f-ad33-4644-9c82-4da4ceaf932f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:15:59.248782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f00c3c3f-ad33-4644-9c82-4da4ceaf932f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:15:59.257586 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:15:59.412490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c7cd4-cabf-4b29-bfef-a145cae99155 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:15:59.433483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c7cd4-cabf-4b29-bfef-a145cae99155 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:15:59.440790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f609d-cc23-49df-adff-baeec60924ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:15:59.441402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f609d-cc23-49df-adff-baeec60924ae service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:15:59.441672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f609d-cc23-49df-adff-baeec60924ae service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:15:59.441934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f609d-cc23-49df-adff-baeec60924ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:15:59.450574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f609d-cc23-49df-adff-baeec60924ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:15:59.452161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f609d-cc23-49df-adff-baeec60924ae service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:15:59.462086 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 756/1516] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:15:59.596875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7a231-5fa1-4087-82f0-621f9379d76e None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:15:59.597120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7a231-5fa1-4087-82f0-621f9379d76e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:15:59.597411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7a231-5fa1-4087-82f0-621f9379d76e None None] PATH_INFO: `/v3` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:15:59.598760 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 761/1517] 10.222.0.64 () {54 vars in 925 bytes} [Fri Sep 23 15:15:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:15:59.605591 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d1d8a-9a35-4267-8461-7dcae875aee9 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:15:59.605880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d1d8a-9a35-4267-8461-7dcae875aee9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:15:59.606144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d1d8a-9a35-4267-8461-7dcae875aee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:15:59.670132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-158d1d8a-9a35-4267-8461-7dcae875aee9 None None] MFA Rules not processed for user `2b8be1ec4a7a45b88d84bd9c48fab407`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:15:59.671346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-158d1d8a-9a35-4267-8461-7dcae875aee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:15:59.689414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:15:59.755685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 757/1518] 10.222.0.64 () {58 vars in 1015 bytes} [Fri Sep 23 15:15:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:16:08.430192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f413d0-e7f8-4849-a230-263788e1c196 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:08.430654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f413d0-e7f8-4849-a230-263788e1c196 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:08.430764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f413d0-e7f8-4849-a230-263788e1c196 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:08.432239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 762/1519] 10.222.0.75 () {54 vars in 961 bytes} [Fri Sep 23 15:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:16:08.439811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbed2a1-d4c1-401b-8928-fd75ef48a31f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:08.440308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbed2a1-d4c1-401b-8928-fd75ef48a31f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:08.440403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbed2a1-d4c1-401b-8928-fd75ef48a31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:08.461533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6dbed2a1-d4c1-401b-8928-fd75ef48a31f None None] MFA Rules not processed for user `2b8be1ec4a7a45b88d84bd9c48fab407`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:16:08.462707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6dbed2a1-d4c1-401b-8928-fd75ef48a31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:08.471511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 758/1520] 10.222.0.75 () {58 vars in 1058 bytes} [Fri Sep 23 15:16:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:16:08.479545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f413d0-e7f8-4849-a230-263788e1c196 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:08.493945 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f413d0-e7f8-4849-a230-263788e1c196 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:08.499277 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858daf7-8d5b-4539-8c4f-dbcdebb68ba4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:08.499868 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858daf7-8d5b-4539-8c4f-dbcdebb68ba4 service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:08.500163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858daf7-8d5b-4539-8c4f-dbcdebb68ba4 service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:08.500431 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858daf7-8d5b-4539-8c4f-dbcdebb68ba4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:08.509069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858daf7-8d5b-4539-8c4f-dbcdebb68ba4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:08.510621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858daf7-8d5b-4539-8c4f-dbcdebb68ba4 service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:08.520300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 763/1521] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:16:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:16:10.993698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e82947-3924-42a6-9913-48ab28693c26 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:10.994344 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e82947-3924-42a6-9913-48ab28693c26 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:10.994344 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e82947-3924-42a6-9913-48ab28693c26 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:10.995607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:16:11.003212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b62c6-01d4-48b3-afc5-7b0c8f55a50f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:11.003984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b62c6-01d4-48b3-afc5-7b0c8f55a50f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:11.004139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b62c6-01d4-48b3-afc5-7b0c8f55a50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:11.026700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1a8b62c6-01d4-48b3-afc5-7b0c8f55a50f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:16:11.027919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1a8b62c6-01d4-48b3-afc5-7b0c8f55a50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:11.036603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:16:11.191274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e82947-3924-42a6-9913-48ab28693c26 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:11.209414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e82947-3924-42a6-9913-48ab28693c26 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:11.215863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dafc15-2af8-4c3d-8ea3-c6dd2ccf7fcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:11.216499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dafc15-2af8-4c3d-8ea3-c6dd2ccf7fcc service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:11.216789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dafc15-2af8-4c3d-8ea3-c6dd2ccf7fcc service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:11.217170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dafc15-2af8-4c3d-8ea3-c6dd2ccf7fcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:11.226122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dafc15-2af8-4c3d-8ea3-c6dd2ccf7fcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:11.227749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dafc15-2af8-4c3d-8ea3-c6dd2ccf7fcc service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:11.237624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 760/1524] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:16:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:16:12.330527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda8af2-9b5b-4cae-961a-881b2eff2922 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:12.331385 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda8af2-9b5b-4cae-961a-881b2eff2922 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:12.331385 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda8af2-9b5b-4cae-961a-881b2eff2922 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:12.333650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 15:16:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:16:12.343029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e977c4-ea62-49c4-97c5-8dea6845b128 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:12.343516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e977c4-ea62-49c4-97c5-8dea6845b128 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:12.343516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e977c4-ea62-49c4-97c5-8dea6845b128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:12.364763 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c1e977c4-ea62-49c4-97c5-8dea6845b128 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:16:12.365985 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c1e977c4-ea62-49c4-97c5-8dea6845b128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:12.374307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:16:12.543633 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda8af2-9b5b-4cae-961a-881b2eff2922 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:12.567419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda8af2-9b5b-4cae-961a-881b2eff2922 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:12.574530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b364a-d366-4c89-a21c-142625574d14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:12.575122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b364a-d366-4c89-a21c-142625574d14 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:12.575391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b364a-d366-4c89-a21c-142625574d14 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:12.575655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b364a-d366-4c89-a21c-142625574d14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:12.584748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b364a-d366-4c89-a21c-142625574d14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:12.586308 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b364a-d366-4c89-a21c-142625574d14 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:12.595709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 766/1527] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:16:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:16:16.935250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dd4ce-c79c-4905-84f0-c3b3800e23c8 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:16.936205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dd4ce-c79c-4905-84f0-c3b3800e23c8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:16.936205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54dd4ce-c79c-4905-84f0-c3b3800e23c8 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:16.938310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:16:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:16:16.947164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1027ec-a894-4135-9535-3df9e4ad7bc8 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:16.947576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1027ec-a894-4135-9535-3df9e4ad7bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:16.947701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1027ec-a894-4135-9535-3df9e4ad7bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:16.969177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-bf1027ec-a894-4135-9535-3df9e4ad7bc8 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:16:16.970416 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-bf1027ec-a894-4135-9535-3df9e4ad7bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:16.979875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:16:17.266426 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54dd4ce-c79c-4905-84f0-c3b3800e23c8 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:17.268307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c54dd4ce-c79c-4905-84f0-c3b3800e23c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:17.349912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54dd4ce-c79c-4905-84f0-c3b3800e23c8 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:17.353924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6d5a7-8a5e-44e3-8bfa-5d6f01efff53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:17.354522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6d5a7-8a5e-44e3-8bfa-5d6f01efff53 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:17.354798 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6d5a7-8a5e-44e3-8bfa-5d6f01efff53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:17.355085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6d5a7-8a5e-44e3-8bfa-5d6f01efff53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:17.364758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6d5a7-8a5e-44e3-8bfa-5d6f01efff53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:17.366189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6d5a7-8a5e-44e3-8bfa-5d6f01efff53 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:17.375823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 763/1530] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:16:19.520855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d333b0a-8655-4cca-8fe0-46b9a8687d68 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:19.521809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d333b0a-8655-4cca-8fe0-46b9a8687d68 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:19.521809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d333b0a-8655-4cca-8fe0-46b9a8687d68 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:19.524056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:16:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:16:19.529683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450439bb-276e-4756-90ca-ad1412856a88 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:19.530146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450439bb-276e-4756-90ca-ad1412856a88 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:19.530259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450439bb-276e-4756-90ca-ad1412856a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:19.550343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-450439bb-276e-4756-90ca-ad1412856a88 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:16:19.551536 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-450439bb-276e-4756-90ca-ad1412856a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:19.560042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:16:19.845282 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d333b0a-8655-4cca-8fe0-46b9a8687d68 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:19.854557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=75 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:16:19.863498 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d333b0a-8655-4cca-8fe0-46b9a8687d68 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:19.867823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc708e8-2917-4591-8019-b05edd1a75df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:19.868407 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc708e8-2917-4591-8019-b05edd1a75df service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:19.868703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc708e8-2917-4591-8019-b05edd1a75df service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:19.868964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc708e8-2917-4591-8019-b05edd1a75df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:19.878303 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc708e8-2917-4591-8019-b05edd1a75df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:19.879736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc708e8-2917-4591-8019-b05edd1a75df service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:19.889987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 769/1533] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:16:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:16:23.646138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5dbdd-4c5d-4f5b-9eaa-a27f65c35365 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:23.646740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5dbdd-4c5d-4f5b-9eaa-a27f65c35365 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:23.646740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5dbdd-4c5d-4f5b-9eaa-a27f65c35365 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:23.648033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:16:23.654682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fb351-844c-4629-b689-97b1fe3ed703 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:23.655138 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fb351-844c-4629-b689-97b1fe3ed703 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:23.655251 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fb351-844c-4629-b689-97b1fe3ed703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:23.680151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-160fb351-844c-4629-b689-97b1fe3ed703 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:16:23.681436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-160fb351-844c-4629-b689-97b1fe3ed703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:23.691004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:16:23.996069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5dbdd-4c5d-4f5b-9eaa-a27f65c35365 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:23.998060 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-16e5dbdd-4c5d-4f5b-9eaa-a27f65c35365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:24.069925 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:16:24.091577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5dbdd-4c5d-4f5b-9eaa-a27f65c35365 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:24.095967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7727b79-ccc6-4eb5-9593-c64f7350203a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:24.096611 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7727b79-ccc6-4eb5-9593-c64f7350203a service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:24.096895 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7727b79-ccc6-4eb5-9593-c64f7350203a service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:24.097158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7727b79-ccc6-4eb5-9593-c64f7350203a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:24.106626 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7727b79-ccc6-4eb5-9593-c64f7350203a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:24.108075 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7727b79-ccc6-4eb5-9593-c64f7350203a service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:24.118511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 766/1536] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:16:26.268142 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2b35c-0b30-4f2e-b1d8-8b2ec5834f36 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:26.269072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2b35c-0b30-4f2e-b1d8-8b2ec5834f36 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:26.269072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2b35c-0b30-4f2e-b1d8-8b2ec5834f36 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:26.271245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 15:16:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:16:26.277796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28803e0-fb54-4b3d-b1c1-de9d60d8f1bf None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:26.278311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28803e0-fb54-4b3d-b1c1-de9d60d8f1bf None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:26.278311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28803e0-fb54-4b3d-b1c1-de9d60d8f1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:26.298306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e28803e0-fb54-4b3d-b1c1-de9d60d8f1bf None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:16:26.299490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e28803e0-fb54-4b3d-b1c1-de9d60d8f1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:26.307875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 15:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:16:26.625015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2b35c-0b30-4f2e-b1d8-8b2ec5834f36 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:16:26.628224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-79f2b35c-0b30-4f2e-b1d8-8b2ec5834f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:16:26.644308 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2b35c-0b30-4f2e-b1d8-8b2ec5834f36 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:16:26.648474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037dd893-507b-4545-bd62-7775f86a2d73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:16:26.649094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dd893-507b-4545-bd62-7775f86a2d73 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:16:26.649360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dd893-507b-4545-bd62-7775f86a2d73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:16:26.649596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037dd893-507b-4545-bd62-7775f86a2d73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:16:26.658832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037dd893-507b-4545-bd62-7775f86a2d73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:16:26.660242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037dd893-507b-4545-bd62-7775f86a2d73 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:16:26.670272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 772/1539] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:16:36.293622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=11 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:16:36.665964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=14 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:51.716928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c897416-cd7a-401c-8efb-e3e75e79a0cf None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:51.719015 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c897416-cd7a-401c-8efb-e3e75e79a0cf None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:51.719015 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c897416-cd7a-401c-8efb-e3e75e79a0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:51.744716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3c897416-cd7a-401c-8efb-e3e75e79a0cf None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:51.746544 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3c897416-cd7a-401c-8efb-e3e75e79a0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:51.762474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 768/1540] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:51.771894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17518795-509f-449d-a5b8-22bde5f3074d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:51.773034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17518795-509f-449d-a5b8-22bde5f3074d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:51.773526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17518795-509f-449d-a5b8-22bde5f3074d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:51.800821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-17518795-509f-449d-a5b8-22bde5f3074d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:51.802388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-17518795-509f-449d-a5b8-22bde5f3074d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:51.813355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 773/1541] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:51.820127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c897416-cd7a-401c-8efb-e3e75e79a0cf None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:51.832893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c897416-cd7a-401c-8efb-e3e75e79a0cf None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:51.840829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076695b-218e-4c23-8c08-ad8dd7fffb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:51.841807 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076695b-218e-4c23-8c08-ad8dd7fffb33 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:51.842296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076695b-218e-4c23-8c08-ad8dd7fffb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:51.842774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076695b-218e-4c23-8c08-ad8dd7fffb33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:51.843651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076695b-218e-4c23-8c08-ad8dd7fffb33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:17:51.844170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076695b-218e-4c23-8c08-ad8dd7fffb33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:51.846160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076695b-218e-4c23-8c08-ad8dd7fffb33 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:51.868204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8001c62-2af7-4b22-978b-0f4d6cfc971c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:51.869118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8001c62-2af7-4b22-978b-0f4d6cfc971c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:51.869624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8001c62-2af7-4b22-978b-0f4d6cfc971c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:51.871239 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 769/1542] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:17:51.892636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076695b-218e-4c23-8c08-ad8dd7fffb33 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:51.910125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a8001c62-2af7-4b22-978b-0f4d6cfc971c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:51.911716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a8001c62-2af7-4b22-978b-0f4d6cfc971c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:51.913714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076695b-218e-4c23-8c08-ad8dd7fffb33 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:51.926498 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 774/1543] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:17:51.928341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea211796-d385-4919-b0ad-5babb5034d36 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:51.928472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac9c0e-fde4-466a-99cf-e97c46c9fa50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:51.929038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea211796-d385-4919-b0ad-5babb5034d36 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:51.929167 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac9c0e-fde4-466a-99cf-e97c46c9fa50 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:51.929424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac9c0e-fde4-466a-99cf-e97c46c9fa50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:51.929697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac9c0e-fde4-466a-99cf-e97c46c9fa50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:51.930047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea211796-d385-4919-b0ad-5babb5034d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:51.930340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac9c0e-fde4-466a-99cf-e97c46c9fa50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:51.931365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-62ac9c0e-fde4-466a-99cf-e97c46c9fa50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:17:51.931365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Sep 23 15:17:51.931365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: Sep 23 15:17:51.932545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac9c0e-fde4-466a-99cf-e97c46c9fa50 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:51.960489 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ea211796-d385-4919-b0ad-5babb5034d36 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:51.961795 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ea211796-d385-4919-b0ad-5babb5034d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:51.972660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 775/1544] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:51.974112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ea810-7e6f-4d3e-b764-0bc05ad03757 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:51.974393 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ea810-7e6f-4d3e-b764-0bc05ad03757 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:51.974662 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ea810-7e6f-4d3e-b764-0bc05ad03757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:51.983880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 770/1545] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:17:51 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:17:51.985310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2340b4a-75c6-40b8-8bc4-3ba5d50d8e3d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:51.985632 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2340b4a-75c6-40b8-8bc4-3ba5d50d8e3d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:51.985899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2340b4a-75c6-40b8-8bc4-3ba5d50d8e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:51.997295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-294ea810-7e6f-4d3e-b764-0bc05ad03757 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:51.998512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-294ea810-7e6f-4d3e-b764-0bc05ad03757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:52.009235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e2340b4a-75c6-40b8-8bc4-3ba5d50d8e3d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:52.010437 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e2340b4a-75c6-40b8-8bc4-3ba5d50d8e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:52.062725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 776/1546] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:52.064187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fd644-f60d-442a-9521-6c25807c709e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.064540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fd644-f60d-442a-9521-6c25807c709e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.064837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fd644-f60d-442a-9521-6c25807c709e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.072501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 771/1547] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:52.074567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e08deb-be47-48cd-94a2-56cc9d3f0e79 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.074842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e08deb-be47-48cd-94a2-56cc9d3f0e79 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.075103 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e08deb-be47-48cd-94a2-56cc9d3f0e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.084702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f81fd644-f60d-442a-9521-6c25807c709e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:52.085895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f81fd644-f60d-442a-9521-6c25807c709e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:52.093574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 777/1548] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:52.094004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d5e08deb-be47-48cd-94a2-56cc9d3f0e79 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:52.094731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2f83f-15e5-4c6e-8598-4c351fa641ea None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.095002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2f83f-15e5-4c6e-8598-4c351fa641ea None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.095208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d5e08deb-be47-48cd-94a2-56cc9d3f0e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:52.095310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2f83f-15e5-4c6e-8598-4c351fa641ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.103462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 772/1549] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:52.104281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e08deb-be47-48cd-94a2-56cc9d3f0e79 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.113963 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e08deb-be47-48cd-94a2-56cc9d3f0e79 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.118579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ecb50-97dc-4fc8-9536-82811c58abaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.119178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ecb50-97dc-4fc8-9536-82811c58abaa admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.119454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ecb50-97dc-4fc8-9536-82811c58abaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.119721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ecb50-97dc-4fc8-9536-82811c58abaa admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.120348 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ecb50-97dc-4fc8-9536-82811c58abaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.121283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-49e2f83f-15e5-4c6e-8598-4c351fa641ea None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:52.122283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ecb50-97dc-4fc8-9536-82811c58abaa admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.122461 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-49e2f83f-15e5-4c6e-8598-4c351fa641ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:52.130374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 778/1550] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:52.131580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363c849-0c6b-4403-bffe-8ba0dd735d5e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.131821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363c849-0c6b-4403-bffe-8ba0dd735d5e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.132090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363c849-0c6b-4403-bffe-8ba0dd735d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.148124 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 773/1551] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/users => generated 296 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:52.148124 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb84b7-be5b-43dd-8289-986f3d6f24d2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.148124 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb84b7-be5b-43dd-8289-986f3d6f24d2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.148124 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb84b7-be5b-43dd-8289-986f3d6f24d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.152706 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7363c849-0c6b-4403-bffe-8ba0dd735d5e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:52.153869 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7363c849-0c6b-4403-bffe-8ba0dd735d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:52.163068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 779/1552] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:52.164277 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e8759-6fef-41dd-bf21-e2917ad4ae85 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.164553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e8759-6fef-41dd-bf21-e2917ad4ae85 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.164853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e8759-6fef-41dd-bf21-e2917ad4ae85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.165969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2beb84b7-be5b-43dd-8289-986f3d6f24d2 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:52.167125 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2beb84b7-be5b-43dd-8289-986f3d6f24d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:52.183638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 774/1553] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:52.183638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694c3c6-1565-4cd9-93a4-acfe90f9d564 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.183638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694c3c6-1565-4cd9-93a4-acfe90f9d564 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.183638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694c3c6-1565-4cd9-93a4-acfe90f9d564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.187899 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f62e8759-6fef-41dd-bf21-e2917ad4ae85 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:52.188944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f62e8759-6fef-41dd-bf21-e2917ad4ae85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:52.198045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 780/1554] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:52.198537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2694c3c6-1565-4cd9-93a4-acfe90f9d564 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:52.199753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2694c3c6-1565-4cd9-93a4-acfe90f9d564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:52.202280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e8759-6fef-41dd-bf21-e2917ad4ae85 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.210461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 775/1555] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:52.211734 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2087b8-8d64-43f5-9b3c-29bb0dffb494 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.212192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2087b8-8d64-43f5-9b3c-29bb0dffb494 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.212459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2087b8-8d64-43f5-9b3c-29bb0dffb494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.213753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e8759-6fef-41dd-bf21-e2917ad4ae85 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.218976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db784f-d80b-4acb-ad94-1df69c4d662b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.219605 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db784f-d80b-4acb-ad94-1df69c4d662b admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.220068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db784f-d80b-4acb-ad94-1df69c4d662b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.220339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db784f-d80b-4acb-ad94-1df69c4d662b admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.221124 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db784f-d80b-4acb-ad94-1df69c4d662b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:17:52.221405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db784f-d80b-4acb-ad94-1df69c4d662b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.222406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-54db784f-d80b-4acb-ad94-1df69c4d662b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:17:52.222406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Sep 23 15:17:52.222406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: Sep 23 15:17:52.223295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db784f-d80b-4acb-ad94-1df69c4d662b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.233640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-dc2087b8-8d64-43f5-9b3c-29bb0dffb494 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:52.234809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dc2087b8-8d64-43f5-9b3c-29bb0dffb494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:52.235666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 781/1556] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:17:52.236238 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db784f-d80b-4acb-ad94-1df69c4d662b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.248034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db784f-d80b-4acb-ad94-1df69c4d662b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.250520 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 776/1557] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:52.251412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2087b8-8d64-43f5-9b3c-29bb0dffb494 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.256111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c2fb7-6ead-4360-b55a-85446b1f02ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.256614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2fb7-6ead-4360-b55a-85446b1f02ec admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.256912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2fb7-6ead-4360-b55a-85446b1f02ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.257184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2fb7-6ead-4360-b55a-85446b1f02ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.257882 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c2fb7-6ead-4360-b55a-85446b1f02ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:17:52.258180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c2fb7-6ead-4360-b55a-85446b1f02ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.259776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c2fb7-6ead-4360-b55a-85446b1f02ec admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.261146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2087b8-8d64-43f5-9b3c-29bb0dffb494 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.265957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e9060-81d9-4f44-a7d4-78fff090bdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.266578 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e9060-81d9-4f44-a7d4-78fff090bdea admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.266851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e9060-81d9-4f44-a7d4-78fff090bdea admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.267121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e9060-81d9-4f44-a7d4-78fff090bdea admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.267805 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e9060-81d9-4f44-a7d4-78fff090bdea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.269431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e9060-81d9-4f44-a7d4-78fff090bdea admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.272529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 782/1558] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:17:52.273799 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af05c6-fc52-497d-b6d9-7f8bedff6b28 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.274144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af05c6-fc52-497d-b6d9-7f8bedff6b28 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.274474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af05c6-fc52-497d-b6d9-7f8bedff6b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.280208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 777/1559] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:52.281468 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add169cc-adad-44a9-abee-9e57da3eb321 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.281749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add169cc-adad-44a9-abee-9e57da3eb321 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.282019 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add169cc-adad-44a9-abee-9e57da3eb321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.297366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c2af05c6-fc52-497d-b6d9-7f8bedff6b28 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:52.298465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c2af05c6-fc52-497d-b6d9-7f8bedff6b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:52.302020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-add169cc-adad-44a9-abee-9e57da3eb321 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:52.303172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-add169cc-adad-44a9-abee-9e57da3eb321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:52.306265 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 783/1560] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:52.307092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af05c6-fc52-497d-b6d9-7f8bedff6b28 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.315837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 778/1561] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:52.316808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af05c6-fc52-497d-b6d9-7f8bedff6b28 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.317139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add169cc-adad-44a9-abee-9e57da3eb321 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.321816 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e214e5-f3a2-4479-95bf-535dd1e846d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.322422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e214e5-f3a2-4479-95bf-535dd1e846d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.322694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e214e5-f3a2-4479-95bf-535dd1e846d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.322968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e214e5-f3a2-4479-95bf-535dd1e846d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.323616 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e214e5-f3a2-4479-95bf-535dd1e846d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:17:52.323913 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e214e5-f3a2-4479-95bf-535dd1e846d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.325529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e214e5-f3a2-4479-95bf-535dd1e846d3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.326960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add169cc-adad-44a9-abee-9e57da3eb321 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.331497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238249dc-74db-4820-8591-c1aa7b1aebcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.332085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238249dc-74db-4820-8591-c1aa7b1aebcf admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.332359 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238249dc-74db-4820-8591-c1aa7b1aebcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.332649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238249dc-74db-4820-8591-c1aa7b1aebcf admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.333300 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238249dc-74db-4820-8591-c1aa7b1aebcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:17:52.333596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238249dc-74db-4820-8591-c1aa7b1aebcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.335211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238249dc-74db-4820-8591-c1aa7b1aebcf admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.337643 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 784/1562] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:17:52.338312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e214e5-f3a2-4479-95bf-535dd1e846d3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.348039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e214e5-f3a2-4479-95bf-535dd1e846d3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.352954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e01c8-46b7-4b51-9949-0a62f9becf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.353568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e01c8-46b7-4b51-9949-0a62f9becf30 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.353834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e01c8-46b7-4b51-9949-0a62f9becf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.354095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e01c8-46b7-4b51-9949-0a62f9becf30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.354582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 779/1563] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:17:52.354716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e01c8-46b7-4b51-9949-0a62f9becf30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.355459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238249dc-74db-4820-8591-c1aa7b1aebcf admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.356783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e01c8-46b7-4b51-9949-0a62f9becf30 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.366169 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238249dc-74db-4820-8591-c1aa7b1aebcf admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.371868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10358eb-66fa-43b0-9a33-b8fbc60262f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.372636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10358eb-66fa-43b0-9a33-b8fbc60262f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.373011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10358eb-66fa-43b0-9a33-b8fbc60262f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.373499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10358eb-66fa-43b0-9a33-b8fbc60262f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.374614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10358eb-66fa-43b0-9a33-b8fbc60262f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:17:52.374918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10358eb-66fa-43b0-9a33-b8fbc60262f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.376696 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10358eb-66fa-43b0-9a33-b8fbc60262f7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.391055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 785/1564] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:52.391055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e01c8-46b7-4b51-9949-0a62f9becf30 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.391324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 780/1565] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:17:52.391324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10358eb-66fa-43b0-9a33-b8fbc60262f7 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.405700 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10358eb-66fa-43b0-9a33-b8fbc60262f7 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.424440 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:52.441876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:52.458324 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e01c8-46b7-4b51-9949-0a62f9becf30 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.462962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e55bf0-354a-486f-b52a-e35a6ecc6310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.463433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828565cc-b9f1-4a14-9cd1-0d610eeecb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.463554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e55bf0-354a-486f-b52a-e35a6ecc6310 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.463845 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e55bf0-354a-486f-b52a-e35a6ecc6310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.464076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828565cc-b9f1-4a14-9cd1-0d610eeecb4f admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.464199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e55bf0-354a-486f-b52a-e35a6ecc6310 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.464351 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828565cc-b9f1-4a14-9cd1-0d610eeecb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.464655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828565cc-b9f1-4a14-9cd1-0d610eeecb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.464773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e55bf0-354a-486f-b52a-e35a6ecc6310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.465329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828565cc-b9f1-4a14-9cd1-0d610eeecb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.466653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e55bf0-354a-486f-b52a-e35a6ecc6310 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.466912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828565cc-b9f1-4a14-9cd1-0d610eeecb4f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.477079 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 781/1566] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:52.477893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828565cc-b9f1-4a14-9cd1-0d610eeecb4f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.487840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828565cc-b9f1-4a14-9cd1-0d610eeecb4f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.492505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3570e-1916-4c84-bf5e-ba0a25acc13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.493126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3570e-1916-4c84-bf5e-ba0a25acc13b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.493402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3570e-1916-4c84-bf5e-ba0a25acc13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.493672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3570e-1916-4c84-bf5e-ba0a25acc13b admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.494316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3570e-1916-4c84-bf5e-ba0a25acc13b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:17:52.494616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3570e-1916-4c84-bf5e-ba0a25acc13b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.496212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3570e-1916-4c84-bf5e-ba0a25acc13b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.496769 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 786/1567] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:52.497660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e55bf0-354a-486f-b52a-e35a6ecc6310 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.511573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 782/1568] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:17:52.512409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3570e-1916-4c84-bf5e-ba0a25acc13b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.521942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3570e-1916-4c84-bf5e-ba0a25acc13b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.569806 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e55bf0-354a-486f-b52a-e35a6ecc6310 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.575212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8688a7c-b6b4-45d0-a99b-332563a452df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.575817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8688a7c-b6b4-45d0-a99b-332563a452df admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.576087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8688a7c-b6b4-45d0-a99b-332563a452df admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.576351 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8688a7c-b6b4-45d0-a99b-332563a452df admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.577036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8688a7c-b6b4-45d0-a99b-332563a452df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:17:52.577327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8688a7c-b6b4-45d0-a99b-332563a452df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.578567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a3dda-344f-4a9a-913f-4e5909090f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.578905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8688a7c-b6b4-45d0-a99b-332563a452df admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.579166 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a3dda-344f-4a9a-913f-4e5909090f7b admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.579435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a3dda-344f-4a9a-913f-4e5909090f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.579695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a3dda-344f-4a9a-913f-4e5909090f7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.580479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a3dda-344f-4a9a-913f-4e5909090f7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.582378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a3dda-344f-4a9a-913f-4e5909090f7b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.590861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 787/1569] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:17:52.591821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8688a7c-b6b4-45d0-a99b-332563a452df admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.601788 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8688a7c-b6b4-45d0-a99b-332563a452df admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.606878 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b4d74-7bb6-4d27-b26c-49db6025fcd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.607475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b4d74-7bb6-4d27-b26c-49db6025fcd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.607746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b4d74-7bb6-4d27-b26c-49db6025fcd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.608015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b4d74-7bb6-4d27-b26c-49db6025fcd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.608670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b4d74-7bb6-4d27-b26c-49db6025fcd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.610554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b4d74-7bb6-4d27-b26c-49db6025fcd3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.617375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 783/1570] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:52.617375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a3dda-344f-4a9a-913f-4e5909090f7b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.640357 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 788/1571] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:52.641237 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b4d74-7bb6-4d27-b26c-49db6025fcd3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.684511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a3dda-344f-4a9a-913f-4e5909090f7b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.703962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b4d74-7bb6-4d27-b26c-49db6025fcd3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.708580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544d3ac-b31d-44bc-8c08-8ad020a186a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.709207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544d3ac-b31d-44bc-8c08-8ad020a186a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.709528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544d3ac-b31d-44bc-8c08-8ad020a186a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.709850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544d3ac-b31d-44bc-8c08-8ad020a186a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.710607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544d3ac-b31d-44bc-8c08-8ad020a186a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.712516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1544d3ac-b31d-44bc-8c08-8ad020a186a8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.718158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:52.739934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 789/1572] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:52.740763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544d3ac-b31d-44bc-8c08-8ad020a186a8 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.746576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa411ff7-42fe-4137-af6b-a9f62a2d16ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.747190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa411ff7-42fe-4137-af6b-a9f62a2d16ef admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.747462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa411ff7-42fe-4137-af6b-a9f62a2d16ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.747726 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa411ff7-42fe-4137-af6b-a9f62a2d16ef admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.748352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa411ff7-42fe-4137-af6b-a9f62a2d16ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.750324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa411ff7-42fe-4137-af6b-a9f62a2d16ef admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.772005 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:52.772582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 784/1573] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/users => generated 302 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:52.774678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa411ff7-42fe-4137-af6b-a9f62a2d16ef admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.785116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa411ff7-42fe-4137-af6b-a9f62a2d16ef admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.805897 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544d3ac-b31d-44bc-8c08-8ad020a186a8 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.810600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0e7db-8374-4077-9e10-31c3ae7cae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.811248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0e7db-8374-4077-9e10-31c3ae7cae39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.811524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0e7db-8374-4077-9e10-31c3ae7cae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.811806 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0e7db-8374-4077-9e10-31c3ae7cae39 admin admin] PATH_INFO: `/v3/projects/1abe4f1508864f77b30c4c635c368f02/users/4cfac62e1c304167abb971815d5da358/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.826635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0e7db-8374-4077-9e10-31c3ae7cae39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1abe4f1508864f77b30c4c635c368f02, user_id=4cfac62e1c304167abb971815d5da358, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.829248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0e7db-8374-4077-9e10-31c3ae7cae39 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.842866 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79e2b7-30a4-4845-baef-6c109803ae8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.843566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e2b7-30a4-4845-baef-6c109803ae8b admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.843910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e2b7-30a4-4845-baef-6c109803ae8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.844251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e2b7-30a4-4845-baef-6c109803ae8b admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.844428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 790/1574] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:52 2022] PUT /identity/v3/projects/1abe4f1508864f77b30c4c635c368f02/users/4cfac62e1c304167abb971815d5da358/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:52.845101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0e7db-8374-4077-9e10-31c3ae7cae39 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.845227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79e2b7-30a4-4845-baef-6c109803ae8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.847336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79e2b7-30a4-4845-baef-6c109803ae8b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.869523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 785/1575] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:52.869523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79e2b7-30a4-4845-baef-6c109803ae8b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.880853 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79e2b7-30a4-4845-baef-6c109803ae8b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.909569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0e7db-8374-4077-9e10-31c3ae7cae39 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:52.914360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2669f7c-bf51-4eb7-abf3-8378ab5442bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.914970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2669f7c-bf51-4eb7-abf3-8378ab5442bb admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.915241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2669f7c-bf51-4eb7-abf3-8378ab5442bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.915509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2669f7c-bf51-4eb7-abf3-8378ab5442bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.916144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2669f7c-bf51-4eb7-abf3-8378ab5442bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.918082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2669f7c-bf51-4eb7-abf3-8378ab5442bb admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.940444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310aa785-006d-4c8d-80cc-2288f99840df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:52.941098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310aa785-006d-4c8d-80cc-2288f99840df admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:52.941369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310aa785-006d-4c8d-80cc-2288f99840df admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:52.941639 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310aa785-006d-4c8d-80cc-2288f99840df admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:52.942295 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310aa785-006d-4c8d-80cc-2288f99840df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:52.944262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310aa785-006d-4c8d-80cc-2288f99840df admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:52.946996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 791/1576] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:52.950218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2669f7c-bf51-4eb7-abf3-8378ab5442bb admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:52.973686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 786/1577] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:52.974504 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310aa785-006d-4c8d-80cc-2288f99840df admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.019268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2669f7c-bf51-4eb7-abf3-8378ab5442bb admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.023719 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:53.043136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310aa785-006d-4c8d-80cc-2288f99840df admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.047900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9c715-8529-40d0-a92e-d4ecf88a8a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.048505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9c715-8529-40d0-a92e-d4ecf88a8a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.048863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9c715-8529-40d0-a92e-d4ecf88a8a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.049123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9c715-8529-40d0-a92e-d4ecf88a8a97 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.049928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9c715-8529-40d0-a92e-d4ecf88a8a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.051827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9c715-8529-40d0-a92e-d4ecf88a8a97 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.061873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:53.076509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 787/1578] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:53.077276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9c715-8529-40d0-a92e-d4ecf88a8a97 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.078718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97511de9-3cb0-4d4f-aca4-ccf87b5890ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.079532 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97511de9-3cb0-4d4f-aca4-ccf87b5890ed admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.079667 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97511de9-3cb0-4d4f-aca4-ccf87b5890ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.079937 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97511de9-3cb0-4d4f-aca4-ccf87b5890ed admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.081206 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97511de9-3cb0-4d4f-aca4-ccf87b5890ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.082343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-97511de9-3cb0-4d4f-aca4-ccf87b5890ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:17:53.082343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Sep 23 15:17:53.082343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: Sep 23 15:17:53.083406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97511de9-3cb0-4d4f-aca4-ccf87b5890ed admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.087491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9c715-8529-40d0-a92e-d4ecf88a8a97 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.091975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb60fc-6e71-4f80-89d1-96d8bd2e7af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.092550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb60fc-6e71-4f80-89d1-96d8bd2e7af5 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.092860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb60fc-6e71-4f80-89d1-96d8bd2e7af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.093122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb60fc-6e71-4f80-89d1-96d8bd2e7af5 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.093740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb60fc-6e71-4f80-89d1-96d8bd2e7af5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.095846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb60fc-6e71-4f80-89d1-96d8bd2e7af5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.110682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 792/1579] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:17:52 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:53.110682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97511de9-3cb0-4d4f-aca4-ccf87b5890ed admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.122892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97511de9-3cb0-4d4f-aca4-ccf87b5890ed admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.123453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 788/1580] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:17:53 2022] POST /identity/v3/users => generated 291 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:53.124884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee21f-b9ce-4ffe-92c9-f1a11567c7bf None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.125167 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee21f-b9ce-4ffe-92c9-f1a11567c7bf None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.125440 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557ee21f-b9ce-4ffe-92c9-f1a11567c7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.130234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bf79f-0918-4834-b322-340e62156355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.130842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bf79f-0918-4834-b322-340e62156355 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.131118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bf79f-0918-4834-b322-340e62156355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.131452 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bf79f-0918-4834-b322-340e62156355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.132101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bf79f-0918-4834-b322-340e62156355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.134975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bf79f-0918-4834-b322-340e62156355 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.145019 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-557ee21f-b9ce-4ffe-92c9-f1a11567c7bf None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:53.148830 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 793/1581] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.148979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-557ee21f-b9ce-4ffe-92c9-f1a11567c7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:53.149088 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bf79f-0918-4834-b322-340e62156355 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.155372 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 789/1582] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:53.156109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ee21f-b9ce-4ffe-92c9-f1a11567c7bf None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.156867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bf79f-0918-4834-b322-340e62156355 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.163414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050dedc-189b-4cf1-89e8-28e8c9f698cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.164297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050dedc-189b-4cf1-89e8-28e8c9f698cf admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.164572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050dedc-189b-4cf1-89e8-28e8c9f698cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.164868 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050dedc-189b-4cf1-89e8-28e8c9f698cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.165522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050dedc-189b-4cf1-89e8-28e8c9f698cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.166420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557ee21f-b9ce-4ffe-92c9-f1a11567c7bf None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.167097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050dedc-189b-4cf1-89e8-28e8c9f698cf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.171080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea3434-1844-44ff-b3ae-fdb61068e4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.171810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea3434-1844-44ff-b3ae-fdb61068e4fc admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.172089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea3434-1844-44ff-b3ae-fdb61068e4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.172370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea3434-1844-44ff-b3ae-fdb61068e4fc admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.173025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea3434-1844-44ff-b3ae-fdb61068e4fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.175112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea3434-1844-44ff-b3ae-fdb61068e4fc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.178592 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 794/1583] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.179229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050dedc-189b-4cf1-89e8-28e8c9f698cf admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.188912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050dedc-189b-4cf1-89e8-28e8c9f698cf admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.194129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29250263-2fa7-493c-bb38-3341fc4d44d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.194798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29250263-2fa7-493c-bb38-3341fc4d44d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.195144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29250263-2fa7-493c-bb38-3341fc4d44d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.195479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29250263-2fa7-493c-bb38-3341fc4d44d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.196190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29250263-2fa7-493c-bb38-3341fc4d44d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.198190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29250263-2fa7-493c-bb38-3341fc4d44d6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.202047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 790/1584] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:17:53 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:53.202047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea3434-1844-44ff-b3ae-fdb61068e4fc admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.213173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea3434-1844-44ff-b3ae-fdb61068e4fc admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.218858 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c83177-2506-4c2c-8c46-dc3102b9c244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.219439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c83177-2506-4c2c-8c46-dc3102b9c244 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.219725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c83177-2506-4c2c-8c46-dc3102b9c244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.219982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c83177-2506-4c2c-8c46-dc3102b9c244 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.220881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c83177-2506-4c2c-8c46-dc3102b9c244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.222536 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c83177-2506-4c2c-8c46-dc3102b9c244 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.223054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 795/1585] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:17:53 2022] POST /identity/v3/users => generated 303 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:53.223897 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29250263-2fa7-493c-bb38-3341fc4d44d6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.233381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29250263-2fa7-493c-bb38-3341fc4d44d6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.234145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 791/1586] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.234958 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c83177-2506-4c2c-8c46-dc3102b9c244 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.240570 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aabc2f-4216-4e24-89d5-09e869dc2606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.241236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aabc2f-4216-4e24-89d5-09e869dc2606 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.241514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aabc2f-4216-4e24-89d5-09e869dc2606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.241928 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aabc2f-4216-4e24-89d5-09e869dc2606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.242452 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aabc2f-4216-4e24-89d5-09e869dc2606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.244163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aabc2f-4216-4e24-89d5-09e869dc2606 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.248004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c83177-2506-4c2c-8c46-dc3102b9c244 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.253595 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d9bff-0ede-4ab5-992c-0ebf0af179b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.254204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d9bff-0ede-4ab5-992c-0ebf0af179b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.254477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d9bff-0ede-4ab5-992c-0ebf0af179b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.254748 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d9bff-0ede-4ab5-992c-0ebf0af179b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.254994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 796/1587] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.255548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d9bff-0ede-4ab5-992c-0ebf0af179b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.255953 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aabc2f-4216-4e24-89d5-09e869dc2606 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.257180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d9bff-0ede-4ab5-992c-0ebf0af179b6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.267139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aabc2f-4216-4e24-89d5-09e869dc2606 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.268030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 792/1588] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.269557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d9bff-0ede-4ab5-992c-0ebf0af179b6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.271505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d91d9bff-0ede-4ab5-992c-0ebf0af179b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:53.272016 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac1883-5228-4646-acc1-bce30182c555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.272643 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac1883-5228-4646-acc1-bce30182c555 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.272923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac1883-5228-4646-acc1-bce30182c555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.273194 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac1883-5228-4646-acc1-bce30182c555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.273835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac1883-5228-4646-acc1-bce30182c555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.275412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac1883-5228-4646-acc1-bce30182c555 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.290457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 797/1589] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.294472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac1883-5228-4646-acc1-bce30182c555 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.305108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac1883-5228-4646-acc1-bce30182c555 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.310505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556d4e7-8bab-4a22-b473-320e00bccf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.311281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556d4e7-8bab-4a22-b473-320e00bccf93 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.311616 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556d4e7-8bab-4a22-b473-320e00bccf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.311987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556d4e7-8bab-4a22-b473-320e00bccf93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.312668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556d4e7-8bab-4a22-b473-320e00bccf93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.314539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556d4e7-8bab-4a22-b473-320e00bccf93 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.325422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 798/1590] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.329497 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556d4e7-8bab-4a22-b473-320e00bccf93 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.336507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d9bff-0ede-4ab5-992c-0ebf0af179b6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.339329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556d4e7-8bab-4a22-b473-320e00bccf93 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.340151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce448924-74f0-4625-844f-534064a8cc82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.340795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce448924-74f0-4625-844f-534064a8cc82 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.341190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce448924-74f0-4625-844f-534064a8cc82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.341463 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce448924-74f0-4625-844f-534064a8cc82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.343939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1486ef2e-c930-4c9b-ab1d-d72f76c634be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.344656 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486ef2e-c930-4c9b-ab1d-d72f76c634be admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.344923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486ef2e-c930-4c9b-ab1d-d72f76c634be admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.345176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1486ef2e-c930-4c9b-ab1d-d72f76c634be admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.346229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1486ef2e-c930-4c9b-ab1d-d72f76c634be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.347834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1486ef2e-c930-4c9b-ab1d-d72f76c634be admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.351140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce448924-74f0-4625-844f-534064a8cc82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.353614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce448924-74f0-4625-844f-534064a8cc82 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.361905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 799/1591] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.363049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1486ef2e-c930-4c9b-ab1d-d72f76c634be admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.366771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 793/1592] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:17:53.369469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce448924-74f0-4625-844f-534064a8cc82 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.373909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1486ef2e-c930-4c9b-ab1d-d72f76c634be admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.379444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683914c-d7fe-442c-970d-32aa349d3b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.380150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683914c-d7fe-442c-970d-32aa349d3b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.380518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683914c-d7fe-442c-970d-32aa349d3b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.380713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce448924-74f0-4625-844f-534064a8cc82 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.380888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683914c-d7fe-442c-970d-32aa349d3b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.381691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683914c-d7fe-442c-970d-32aa349d3b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.383259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683914c-d7fe-442c-970d-32aa349d3b89 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.386099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405aceb-39e7-4d59-9bcd-c16eadcc74c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.386696 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405aceb-39e7-4d59-9bcd-c16eadcc74c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.386977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405aceb-39e7-4d59-9bcd-c16eadcc74c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.387254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405aceb-39e7-4d59-9bcd-c16eadcc74c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.388045 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405aceb-39e7-4d59-9bcd-c16eadcc74c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.390362 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405aceb-39e7-4d59-9bcd-c16eadcc74c1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.396845 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 800/1593] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.397703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683914c-d7fe-442c-970d-32aa349d3b89 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.404711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 794/1594] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.404711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405aceb-39e7-4d59-9bcd-c16eadcc74c1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.409200 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683914c-d7fe-442c-970d-32aa349d3b89 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.414675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e2381-232c-424b-8d7a-7ad6f0cc17e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.415890 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e2381-232c-424b-8d7a-7ad6f0cc17e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.416289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e2381-232c-424b-8d7a-7ad6f0cc17e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.416563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e2381-232c-424b-8d7a-7ad6f0cc17e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.417553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e2381-232c-424b-8d7a-7ad6f0cc17e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.419114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405aceb-39e7-4d59-9bcd-c16eadcc74c1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.419281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e2381-232c-424b-8d7a-7ad6f0cc17e4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.424007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b4a0d-d38f-42f0-9272-950418d4de40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.424807 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b4a0d-d38f-42f0-9272-950418d4de40 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.425069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b4a0d-d38f-42f0-9272-950418d4de40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.425357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b4a0d-d38f-42f0-9272-950418d4de40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.426124 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b4a0d-d38f-42f0-9272-950418d4de40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.427730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b4a0d-d38f-42f0-9272-950418d4de40 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.430251 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 801/1595] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.430649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e2381-232c-424b-8d7a-7ad6f0cc17e4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.443298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 795/1596] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.443298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b4a0d-d38f-42f0-9272-950418d4de40 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.443607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e2381-232c-424b-8d7a-7ad6f0cc17e4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.448402 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276541b-3bd8-43f7-86bc-7ccc9573be23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.449179 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276541b-3bd8-43f7-86bc-7ccc9573be23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.449516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276541b-3bd8-43f7-86bc-7ccc9573be23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.450004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276541b-3bd8-43f7-86bc-7ccc9573be23 admin admin] PATH_INFO: `/v3/projects/02c0792b1529441b89b3206da430c7de/users/92dde467c0bc4f82a147e2b473adb68c/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.455043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b4a0d-d38f-42f0-9272-950418d4de40 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.460612 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad62563-6774-4b0f-ae37-d59c3644afb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.461373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad62563-6774-4b0f-ae37-d59c3644afb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.461965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad62563-6774-4b0f-ae37-d59c3644afb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.461965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad62563-6774-4b0f-ae37-d59c3644afb5 admin admin] PATH_INFO: `/v3/projects/bb9f84f2d16a4389a4be567cd2be3a8a/users/e38b853b47b64567b00092eb32201b4d/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.469759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276541b-3bd8-43f7-86bc-7ccc9573be23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c0792b1529441b89b3206da430c7de, user_id=92dde467c0bc4f82a147e2b473adb68c, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.472909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276541b-3bd8-43f7-86bc-7ccc9573be23 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.480121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad62563-6774-4b0f-ae37-d59c3644afb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb9f84f2d16a4389a4be567cd2be3a8a, user_id=e38b853b47b64567b00092eb32201b4d, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.481191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-2ad62563-6774-4b0f-ae37-d59c3644afb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:17:53.481191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Sep 23 15:17:53.481191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: Sep 23 15:17:53.483114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad62563-6774-4b0f-ae37-d59c3644afb5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.490751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 802/1597] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:53 2022] PUT /identity/v3/projects/02c0792b1529441b89b3206da430c7de/users/92dde467c0bc4f82a147e2b473adb68c/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:53.491552 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276541b-3bd8-43f7-86bc-7ccc9573be23 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.509348 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 796/1598] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:53 2022] PUT /identity/v3/projects/bb9f84f2d16a4389a4be567cd2be3a8a/users/e38b853b47b64567b00092eb32201b4d/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:53.510223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad62563-6774-4b0f-ae37-d59c3644afb5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.554101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:53.578666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276541b-3bd8-43f7-86bc-7ccc9573be23 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.588956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:53.606822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad62563-6774-4b0f-ae37-d59c3644afb5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.613255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02dbf5-c151-4420-9f49-62a9f4e81f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.614005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02dbf5-c151-4420-9f49-62a9f4e81f11 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.614349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02dbf5-c151-4420-9f49-62a9f4e81f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.614715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02dbf5-c151-4420-9f49-62a9f4e81f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.615432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02dbf5-c151-4420-9f49-62a9f4e81f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.617205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02dbf5-c151-4420-9f49-62a9f4e81f11 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.628540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 797/1599] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.629271 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02dbf5-c151-4420-9f49-62a9f4e81f11 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.639321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02dbf5-c151-4420-9f49-62a9f4e81f11 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.644970 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc71273-907c-4439-af39-b97eeae41d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.645796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc71273-907c-4439-af39-b97eeae41d20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.646132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc71273-907c-4439-af39-b97eeae41d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.646465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc71273-907c-4439-af39-b97eeae41d20 admin admin] PATH_INFO: `/v3/projects/f3f04aa2dcf84d32bcf7c8cebf64feee/users/811321b5291b4b8f97f0a047805a1d8d/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.649223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df230b-9352-4b2b-807e-339b6cddca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.649960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df230b-9352-4b2b-807e-339b6cddca27 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.650231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df230b-9352-4b2b-807e-339b6cddca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.650538 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df230b-9352-4b2b-807e-339b6cddca27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.651341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df230b-9352-4b2b-807e-339b6cddca27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.653052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df230b-9352-4b2b-807e-339b6cddca27 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.663400 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc71273-907c-4439-af39-b97eeae41d20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f04aa2dcf84d32bcf7c8cebf64feee, user_id=811321b5291b4b8f97f0a047805a1d8d, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.666085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc71273-907c-4439-af39-b97eeae41d20 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.670187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 803/1600] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:53.673576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df230b-9352-4b2b-807e-339b6cddca27 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.689516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 798/1601] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:53 2022] PUT /identity/v3/projects/f3f04aa2dcf84d32bcf7c8cebf64feee/users/811321b5291b4b8f97f0a047805a1d8d/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:53.690449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc71273-907c-4439-af39-b97eeae41d20 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.693245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df230b-9352-4b2b-807e-339b6cddca27 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.765336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da484129-2c4e-4138-81ca-5eb2274935fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.765875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da484129-2c4e-4138-81ca-5eb2274935fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.766150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da484129-2c4e-4138-81ca-5eb2274935fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.766440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da484129-2c4e-4138-81ca-5eb2274935fb admin admin] PATH_INFO: `/v3/projects/b0f632ecb8ee4e6fb59978560789f599/users/a568b6fd614744abaf5ca41fb26c1f13/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.773189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc71273-907c-4439-af39-b97eeae41d20 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.778643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc91e7-2f2f-46c5-8e2d-acbd26473522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.779324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc91e7-2f2f-46c5-8e2d-acbd26473522 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.779867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc91e7-2f2f-46c5-8e2d-acbd26473522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.780140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc91e7-2f2f-46c5-8e2d-acbd26473522 admin admin] PATH_INFO: `/v3/projects/dca8c512d6184ee2a57d7ca823d337e8/users/28afc40bcc574533862417e739e366b2/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.783558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da484129-2c4e-4138-81ca-5eb2274935fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f632ecb8ee4e6fb59978560789f599, user_id=a568b6fd614744abaf5ca41fb26c1f13, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.786349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da484129-2c4e-4138-81ca-5eb2274935fb admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.803158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc91e7-2f2f-46c5-8e2d-acbd26473522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dca8c512d6184ee2a57d7ca823d337e8, user_id=28afc40bcc574533862417e739e366b2, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.806157 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 804/1602] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:53 2022] PUT /identity/v3/projects/b0f632ecb8ee4e6fb59978560789f599/users/a568b6fd614744abaf5ca41fb26c1f13/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:53.807322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87fba3-3dca-4a23-a630-659bd37ef5ee None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.807602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87fba3-3dca-4a23-a630-659bd37ef5ee None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.807866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87fba3-3dca-4a23-a630-659bd37ef5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.810412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc91e7-2f2f-46c5-8e2d-acbd26473522 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.828190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 799/1603] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:53 2022] PUT /identity/v3/projects/dca8c512d6184ee2a57d7ca823d337e8/users/28afc40bcc574533862417e739e366b2/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:53.830054 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ab25f-8c0a-42c1-af40-cf8f45d6fae6 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.830616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ab25f-8c0a-42c1-af40-cf8f45d6fae6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.831057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ab25f-8c0a-42c1-af40-cf8f45d6fae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.840131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1a87fba3-3dca-4a23-a630-659bd37ef5ee None None] MFA Rules not processed for user `92dde467c0bc4f82a147e2b473adb68c`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:53.841462 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1a87fba3-3dca-4a23-a630-659bd37ef5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:53.856955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-374ab25f-8c0a-42c1-af40-cf8f45d6fae6 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:53.858475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-374ab25f-8c0a-42c1-af40-cf8f45d6fae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:53.920501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 805/1604] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:53 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:53.921682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87fba3-3dca-4a23-a630-659bd37ef5ee None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.932218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 800/1605] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:53.933259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ab25f-8c0a-42c1-af40-cf8f45d6fae6 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:53.934311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87fba3-3dca-4a23-a630-659bd37ef5ee None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.939388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05addd39-15b1-4a63-9bac-0fe43b0ce173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.940856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05addd39-15b1-4a63-9bac-0fe43b0ce173 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.941453 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05addd39-15b1-4a63-9bac-0fe43b0ce173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.941997 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05addd39-15b1-4a63-9bac-0fe43b0ce173 admin admin] PATH_INFO: `/v3/projects/a443d1afb1d043e08d6ab0320d4fe92a/users/23f67753ed624f4a88930d55a48f2d81/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.945143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ab25f-8c0a-42c1-af40-cf8f45d6fae6 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:53.951259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c9aee-fa87-4056-87c0-fed3c52a81f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:53.952242 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c9aee-fa87-4056-87c0-fed3c52a81f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.952708 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c9aee-fa87-4056-87c0-fed3c52a81f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.953204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c9aee-fa87-4056-87c0-fed3c52a81f6 admin admin] PATH_INFO: `/v3/projects/d4fd7ca907fa4806904e1c00b21b5447/users/4bded8b580f34e08b2890eb3430f4b80/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.961900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05addd39-15b1-4a63-9bac-0fe43b0ce173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a443d1afb1d043e08d6ab0320d4fe92a, user_id=23f67753ed624f4a88930d55a48f2d81, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.965169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05addd39-15b1-4a63-9bac-0fe43b0ce173 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.969545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:53.970939 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c9aee-fa87-4056-87c0-fed3c52a81f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4fd7ca907fa4806904e1c00b21b5447, user_id=4bded8b580f34e08b2890eb3430f4b80, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:53.975184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c9aee-fa87-4056-87c0-fed3c52a81f6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:53.978705 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:53.984404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 806/1606] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:53 2022] PUT /identity/v3/projects/a443d1afb1d043e08d6ab0320d4fe92a/users/23f67753ed624f4a88930d55a48f2d81/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:53.986052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887d4ac-314f-4db0-bca3-a84a6e1146d3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.986513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887d4ac-314f-4db0-bca3-a84a6e1146d3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.986889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887d4ac-314f-4db0-bca3-a84a6e1146d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:53.994843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 801/1607] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:53 2022] PUT /identity/v3/projects/d4fd7ca907fa4806904e1c00b21b5447/users/4bded8b580f34e08b2890eb3430f4b80/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:53.996381 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b9fd6-d2ae-4804-a5e2-6c7e979407c2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:53.996829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b9fd6-d2ae-4804-a5e2-6c7e979407c2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:53.997175 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b9fd6-d2ae-4804-a5e2-6c7e979407c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.016485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f80b9fd6-d2ae-4804-a5e2-6c7e979407c2 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:54.017916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f80b9fd6-d2ae-4804-a5e2-6c7e979407c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:54.022539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b887d4ac-314f-4db0-bca3-a84a6e1146d3 None None] MFA Rules not processed for user `811321b5291b4b8f97f0a047805a1d8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:54.023732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b887d4ac-314f-4db0-bca3-a84a6e1146d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:54.095794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 802/1608] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:54.097040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0460b8f-090d-4a39-94e9-814afe87758a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.097456 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0460b8f-090d-4a39-94e9-814afe87758a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.098791 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0460b8f-090d-4a39-94e9-814afe87758a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.104695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 807/1609] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:54.104695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887d4ac-314f-4db0-bca3-a84a6e1146d3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:54.116743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887d4ac-314f-4db0-bca3-a84a6e1146d3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:54.122314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345a867-9110-4657-b34b-a001755dbdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:54.122971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345a867-9110-4657-b34b-a001755dbdb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.123620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345a867-9110-4657-b34b-a001755dbdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.123620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345a867-9110-4657-b34b-a001755dbdb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.124269 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345a867-9110-4657-b34b-a001755dbdb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:54.126632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7345a867-9110-4657-b34b-a001755dbdb4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:54.166935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d0460b8f-090d-4a39-94e9-814afe87758a None None] MFA Rules not processed for user `28afc40bcc574533862417e739e366b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:54.168093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d0460b8f-090d-4a39-94e9-814afe87758a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:54.185747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 808/1610] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:17:54 2022] POST /identity/v3/projects => generated 401 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:54.187085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345a867-9110-4657-b34b-a001755dbdb4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:54.281048 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 803/1611] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:54.281048 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717878a-884b-42a9-bc77-ee45acef5892 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.281048 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717878a-884b-42a9-bc77-ee45acef5892 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.281048 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717878a-884b-42a9-bc77-ee45acef5892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.290132 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345a867-9110-4657-b34b-a001755dbdb4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:54.295941 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91852d3-52dd-4beb-9649-c36c6d826319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:54.297105 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91852d3-52dd-4beb-9649-c36c6d826319 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.297614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91852d3-52dd-4beb-9649-c36c6d826319 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.298163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91852d3-52dd-4beb-9649-c36c6d826319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.304182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9717878a-884b-42a9-bc77-ee45acef5892 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:54.305944 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9717878a-884b-42a9-bc77-ee45acef5892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:54.380097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91852d3-52dd-4beb-9649-c36c6d826319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:54.381509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91852d3-52dd-4beb-9649-c36c6d826319 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:54.397081 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 804/1612] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:54.397300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 809/1613] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:17:54.397300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91852d3-52dd-4beb-9649-c36c6d826319 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:54.397510 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be5ce7-1175-4b40-8fdb-1299b5ae78bf None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.397510 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be5ce7-1175-4b40-8fdb-1299b5ae78bf None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.397510 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be5ce7-1175-4b40-8fdb-1299b5ae78bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.405616 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91852d3-52dd-4beb-9649-c36c6d826319 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:54.411933 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473f1de-8ae6-4c32-9ca1-df9e2dbc1977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:54.412570 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473f1de-8ae6-4c32-9ca1-df9e2dbc1977 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.412873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473f1de-8ae6-4c32-9ca1-df9e2dbc1977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.413145 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473f1de-8ae6-4c32-9ca1-df9e2dbc1977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.425412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473f1de-8ae6-4c32-9ca1-df9e2dbc1977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:54.426790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-07be5ce7-1175-4b40-8fdb-1299b5ae78bf None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:54.427123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473f1de-8ae6-4c32-9ca1-df9e2dbc1977 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:54.428790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-07be5ce7-1175-4b40-8fdb-1299b5ae78bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:54.433145 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:54.441377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 810/1614] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:17:54.442475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 805/1615] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:54.444035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be5ce7-1175-4b40-8fdb-1299b5ae78bf None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:54.448057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4b80-86a8-4e1b-8f7f-5dd00a6deff2 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.448341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4b80-86a8-4e1b-8f7f-5dd00a6deff2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.448665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4b80-86a8-4e1b-8f7f-5dd00a6deff2 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.450823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 811/1616] 10.222.0.75 () {54 vars in 961 bytes} [Fri Sep 23 15:17:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:17:54.451879 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d4b80-86a8-4e1b-8f7f-5dd00a6deff2 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:54.458606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be5ce7-1175-4b40-8fdb-1299b5ae78bf None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:54.466120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517b2ca-5e7f-412a-86de-e8d641fac268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:54.466886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517b2ca-5e7f-412a-86de-e8d641fac268 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.467278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517b2ca-5e7f-412a-86de-e8d641fac268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.467509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d4b80-86a8-4e1b-8f7f-5dd00a6deff2 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:54.467717 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517b2ca-5e7f-412a-86de-e8d641fac268 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.468454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517b2ca-5e7f-412a-86de-e8d641fac268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:54.472057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517b2ca-5e7f-412a-86de-e8d641fac268 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:54.474808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcceb0-3ef8-46d0-98df-41ad42c24326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:54.475514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcceb0-3ef8-46d0-98df-41ad42c24326 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.475794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcceb0-3ef8-46d0-98df-41ad42c24326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.476064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcceb0-3ef8-46d0-98df-41ad42c24326 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.477004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcceb0-3ef8-46d0-98df-41ad42c24326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:54.479517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcceb0-3ef8-46d0-98df-41ad42c24326 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:54.506265 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 806/1617] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:17:54 2022] POST /identity/v3/users => generated 307 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:54.507312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517b2ca-5e7f-412a-86de-e8d641fac268 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:54.513812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:54.520724 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517b2ca-5e7f-412a-86de-e8d641fac268 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:54.522595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 812/1618] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:17:54 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:54.527310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcceb0-3ef8-46d0-98df-41ad42c24326 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:54.601965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266cd2c-d964-48ab-b0a0-1522c3bd7fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:54.602683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266cd2c-d964-48ab-b0a0-1522c3bd7fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.603039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266cd2c-d964-48ab-b0a0-1522c3bd7fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.603378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266cd2c-d964-48ab-b0a0-1522c3bd7fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.609389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcceb0-3ef8-46d0-98df-41ad42c24326 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:54.615204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da054f-a738-48f9-9616-bfcd986a37e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:54.616257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da054f-a738-48f9-9616-bfcd986a37e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.616740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da054f-a738-48f9-9616-bfcd986a37e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.617209 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da054f-a738-48f9-9616-bfcd986a37e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.697550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266cd2c-d964-48ab-b0a0-1522c3bd7fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:54.699213 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266cd2c-d964-48ab-b0a0-1522c3bd7fc1 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:54.711025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da054f-a738-48f9-9616-bfcd986a37e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:54.714795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 807/1619] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:17:54.716352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c272649-2e36-4c53-bc1d-c08c70dc216b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.716792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c272649-2e36-4c53-bc1d-c08c70dc216b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.717079 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c272649-2e36-4c53-bc1d-c08c70dc216b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.726366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da054f-a738-48f9-9616-bfcd986a37e5 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:54.766617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 813/1620] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:17:54.772343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da054f-a738-48f9-9616-bfcd986a37e5 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:54.785845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9c272649-2e36-4c53-bc1d-c08c70dc216b None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:54.787781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9c272649-2e36-4c53-bc1d-c08c70dc216b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:54.805901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da054f-a738-48f9-9616-bfcd986a37e5 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:54.819994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e29e6d-886a-4204-8ae7-f14df2982c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:54.820903 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e29e6d-886a-4204-8ae7-f14df2982c77 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.821444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e29e6d-886a-4204-8ae7-f14df2982c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.821444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e29e6d-886a-4204-8ae7-f14df2982c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.822114 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e29e6d-886a-4204-8ae7-f14df2982c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:54.825727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e29e6d-886a-4204-8ae7-f14df2982c77 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:54.859402 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 814/1621] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:54.859960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e29e6d-886a-4204-8ae7-f14df2982c77 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:54.889808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e29e6d-886a-4204-8ae7-f14df2982c77 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:54.902008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 808/1622] 10.222.0.75 () {58 vars in 1058 bytes} [Fri Sep 23 15:17:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:54.903056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38fce9-496a-45aa-81fa-caff4e794fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:54.903201 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c272649-2e36-4c53-bc1d-c08c70dc216b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:54.903990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38fce9-496a-45aa-81fa-caff4e794fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.904436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38fce9-496a-45aa-81fa-caff4e794fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.904905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c38fce9-496a-45aa-81fa-caff4e794fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.905737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38fce9-496a-45aa-81fa-caff4e794fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:54.907871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c38fce9-496a-45aa-81fa-caff4e794fa9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:54.916955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c272649-2e36-4c53-bc1d-c08c70dc216b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:54.925092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b287c-9a26-4e95-b34a-e7cb48620d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:54.926535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b287c-9a26-4e95-b34a-e7cb48620d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.927009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b287c-9a26-4e95-b34a-e7cb48620d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.928448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b287c-9a26-4e95-b34a-e7cb48620d9d admin admin] PATH_INFO: `/v3/projects/6d6c79119ed741b88ca185ca2af3fa61/users/baedced839a6484cb9ac702bb2055c59/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:54.960883 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 815/1623] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:17:54 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:54.961911 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38fce9-496a-45aa-81fa-caff4e794fa9 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:54.962199 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b287c-9a26-4e95-b34a-e7cb48620d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d6c79119ed741b88ca185ca2af3fa61, user_id=baedced839a6484cb9ac702bb2055c59, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:54.969842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b287c-9a26-4e95-b34a-e7cb48620d9d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:54.976044 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c38fce9-496a-45aa-81fa-caff4e794fa9 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:54.982248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d952e06-5311-404b-b884-7e921c21f8b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:54.983495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d952e06-5311-404b-b884-7e921c21f8b7 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:54.984861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d952e06-5311-404b-b884-7e921c21f8b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:54.985239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d952e06-5311-404b-b884-7e921c21f8b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:55.003058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 809/1624] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:54 2022] PUT /identity/v3/projects/6d6c79119ed741b88ca185ca2af3fa61/users/baedced839a6484cb9ac702bb2055c59/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:55.008837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b287c-9a26-4e95-b34a-e7cb48620d9d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:55.028223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:55.153944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d952e06-5311-404b-b884-7e921c21f8b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:55.156260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d952e06-5311-404b-b884-7e921c21f8b7 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:55.161830 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:55.180716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 816/1625] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:17:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:17:55.181567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d952e06-5311-404b-b884-7e921c21f8b7 service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:55.186133 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b287c-9a26-4e95-b34a-e7cb48620d9d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:55.194233 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d952e06-5311-404b-b884-7e921c21f8b7 service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:55.198652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3462b8-a9bf-4559-abf9-2ed5d7d22c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:55.200933 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3462b8-a9bf-4559-abf9-2ed5d7d22c28 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:55.201347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3462b8-a9bf-4559-abf9-2ed5d7d22c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:55.201535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3462b8-a9bf-4559-abf9-2ed5d7d22c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:55.203168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3462b8-a9bf-4559-abf9-2ed5d7d22c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:55.204783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d72d67-1bb8-47d3-971b-40fa4e4a78a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:55.206706 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d72d67-1bb8-47d3-971b-40fa4e4a78a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:55.207145 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d72d67-1bb8-47d3-971b-40fa4e4a78a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:55.207388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d72d67-1bb8-47d3-971b-40fa4e4a78a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:55.208121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d72d67-1bb8-47d3-971b-40fa4e4a78a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:55.209312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3462b8-a9bf-4559-abf9-2ed5d7d22c28 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:55.210319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d72d67-1bb8-47d3-971b-40fa4e4a78a5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:55.226661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 817/1626] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:55.231061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d72d67-1bb8-47d3-971b-40fa4e4a78a5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:55.247799 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d72d67-1bb8-47d3-971b-40fa4e4a78a5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:55.251401 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 810/1627] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:55.257844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3462b8-a9bf-4559-abf9-2ed5d7d22c28 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:55.265221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527d7da-f00f-4982-b3aa-ce4d3c608e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:55.266797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527d7da-f00f-4982-b3aa-ce4d3c608e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:55.267539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527d7da-f00f-4982-b3aa-ce4d3c608e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:55.268845 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527d7da-f00f-4982-b3aa-ce4d3c608e0c admin admin] PATH_INFO: `/v3/projects/6d6c79119ed741b88ca185ca2af3fa61/users/baedced839a6484cb9ac702bb2055c59/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:55.271984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527d7da-f00f-4982-b3aa-ce4d3c608e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d6c79119ed741b88ca185ca2af3fa61, user_id=baedced839a6484cb9ac702bb2055c59, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:55.275713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527d7da-f00f-4982-b3aa-ce4d3c608e0c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:55.295767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 818/1628] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:55 2022] PUT /identity/v3/projects/6d6c79119ed741b88ca185ca2af3fa61/users/baedced839a6484cb9ac702bb2055c59/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:17:55.308986 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57539426-f28f-4b60-b7f5-bf36837496ae None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:55.309603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57539426-f28f-4b60-b7f5-bf36837496ae None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:55.310032 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57539426-f28f-4b60-b7f5-bf36837496ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:55.317508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3462b8-a9bf-4559-abf9-2ed5d7d22c28 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:55.380784 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-57539426-f28f-4b60-b7f5-bf36837496ae None None] MFA Rules not processed for user `baedced839a6484cb9ac702bb2055c59`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:55.386154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-57539426-f28f-4b60-b7f5-bf36837496ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:55.541093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f6844-b7cd-4498-8081-8248f1e3b5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:55.541093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f6844-b7cd-4498-8081-8248f1e3b5f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:55.541093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f6844-b7cd-4498-8081-8248f1e3b5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:55.541093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19f6844-b7cd-4498-8081-8248f1e3b5f3 admin admin] PATH_INFO: `/v3/projects/44c98da3939946e8ad894efd1e56c5a7/users/ef55537360174d89ad12fa6a7354b407/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:55.565925 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f6844-b7cd-4498-8081-8248f1e3b5f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c98da3939946e8ad894efd1e56c5a7, user_id=ef55537360174d89ad12fa6a7354b407, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:55.571641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19f6844-b7cd-4498-8081-8248f1e3b5f3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:55.625173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 811/1629] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:55 2022] PUT /identity/v3/projects/44c98da3939946e8ad894efd1e56c5a7/users/ef55537360174d89ad12fa6a7354b407/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:55.630140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f6844-b7cd-4498-8081-8248f1e3b5f3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:55.653397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 819/1630] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:55 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:55.669323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57539426-f28f-4b60-b7f5-bf36837496ae None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:55.780223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57539426-f28f-4b60-b7f5-bf36837496ae None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:55.812523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19f6844-b7cd-4498-8081-8248f1e3b5f3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:55.828303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a8862-43a1-411d-836f-cc1644a20848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:55.828963 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a8862-43a1-411d-836f-cc1644a20848 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:55.829253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a8862-43a1-411d-836f-cc1644a20848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:55.829524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a8862-43a1-411d-836f-cc1644a20848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:55.830228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a8862-43a1-411d-836f-cc1644a20848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:55.832040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a8862-43a1-411d-836f-cc1644a20848 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:55.858448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 812/1631] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:55.863972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a8862-43a1-411d-836f-cc1644a20848 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:55.899376 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a8862-43a1-411d-836f-cc1644a20848 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:55.907522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c105422-a84e-4ec0-a93c-db6b3880d537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:55.909655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c105422-a84e-4ec0-a93c-db6b3880d537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:55.910216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c105422-a84e-4ec0-a93c-db6b3880d537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:55.910771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c105422-a84e-4ec0-a93c-db6b3880d537 admin admin] PATH_INFO: `/v3/projects/44c98da3939946e8ad894efd1e56c5a7/users/ef55537360174d89ad12fa6a7354b407/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:55.913574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c105422-a84e-4ec0-a93c-db6b3880d537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c98da3939946e8ad894efd1e56c5a7, user_id=ef55537360174d89ad12fa6a7354b407, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:55.918906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c105422-a84e-4ec0-a93c-db6b3880d537 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:55.924679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb9181-b59f-42bc-b5cb-89e141a67ca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:55.925974 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb9181-b59f-42bc-b5cb-89e141a67ca0 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:55.926402 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb9181-b59f-42bc-b5cb-89e141a67ca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:55.926840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb9181-b59f-42bc-b5cb-89e141a67ca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:55.944821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 813/1632] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:55 2022] PUT /identity/v3/projects/44c98da3939946e8ad894efd1e56c5a7/users/ef55537360174d89ad12fa6a7354b407/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:55.946733 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0925f-9fa6-4766-92da-e39ca39081e7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:55.947303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0925f-9fa6-4766-92da-e39ca39081e7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:55.947667 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0925f-9fa6-4766-92da-e39ca39081e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:55.982195 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:56.008108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-56c0925f-9fa6-4766-92da-e39ca39081e7 None None] MFA Rules not processed for user `ef55537360174d89ad12fa6a7354b407`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:56.009839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-56c0925f-9fa6-4766-92da-e39ca39081e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:56.027080 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:56.125564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb9181-b59f-42bc-b5cb-89e141a67ca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:56.130271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb9181-b59f-42bc-b5cb-89e141a67ca0 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:56.177542 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 820/1633] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:17:55 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:17:56.211333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 814/1634] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:56.228763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb9181-b59f-42bc-b5cb-89e141a67ca0 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:56.247224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb9181-b59f-42bc-b5cb-89e141a67ca0 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:56.337664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b52099-56ec-4f66-bde0-1cffdbe1db4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:56.338424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b52099-56ec-4f66-bde0-1cffdbe1db4a service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:56.338777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b52099-56ec-4f66-bde0-1cffdbe1db4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:56.340560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b52099-56ec-4f66-bde0-1cffdbe1db4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:56.361767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b52099-56ec-4f66-bde0-1cffdbe1db4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:56.367707 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b52099-56ec-4f66-bde0-1cffdbe1db4a service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:56.397135 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 821/1635] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:17:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:17:56.503901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0925f-9fa6-4766-92da-e39ca39081e7 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:56.602817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bb279-c95d-44b7-b41a-386613d18f6d None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:56.602817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bb279-c95d-44b7-b41a-386613d18f6d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:56.602817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bb279-c95d-44b7-b41a-386613d18f6d None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:56.602817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 822/1636] 10.222.0.75 () {54 vars in 961 bytes} [Fri Sep 23 15:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:17:56.621647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664a524-1444-4a80-9fca-74f5218aabdf None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:56.621926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664a524-1444-4a80-9fca-74f5218aabdf None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:56.622199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664a524-1444-4a80-9fca-74f5218aabdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:56.662951 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c664a524-1444-4a80-9fca-74f5218aabdf None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:56.664643 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c664a524-1444-4a80-9fca-74f5218aabdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:56.682655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0925f-9fa6-4766-92da-e39ca39081e7 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:56.688752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c434e8a-7e22-4d5a-854b-5f8e132303dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:56.698686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c434e8a-7e22-4d5a-854b-5f8e132303dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:56.698980 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c434e8a-7e22-4d5a-854b-5f8e132303dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:56.699230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c434e8a-7e22-4d5a-854b-5f8e132303dd admin admin] PATH_INFO: `/v3/users/92dde467c0bc4f82a147e2b473adb68c` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:56.702312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c434e8a-7e22-4d5a-854b-5f8e132303dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=92dde467c0bc4f82a147e2b473adb68c)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:56.705252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: WARNING py.warnings [None req-6c434e8a-7e22-4d5a-854b-5f8e132303dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:17:56.705252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: warnings.warn(msg) Sep 23 15:17:56.705252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: Sep 23 15:17:56.706166 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c434e8a-7e22-4d5a-854b-5f8e132303dd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:56.789073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 823/1637] 10.222.0.75 () {58 vars in 1058 bytes} [Fri Sep 23 15:17:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:56.795490 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27bb32-323a-416e-9c8f-56e2313273cc None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:56.796566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27bb32-323a-416e-9c8f-56e2313273cc None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:56.797290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27bb32-323a-416e-9c8f-56e2313273cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:56.842695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6c434e8a-7e22-4d5a-854b-5f8e132303dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92dde467c0bc4f82a147e2b473adb68c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:56.844623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6c434e8a-7e22-4d5a-854b-5f8e132303dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92dde467c0bc4f82a147e2b473adb68c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:56.874940 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6c434e8a-7e22-4d5a-854b-5f8e132303dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92dde467c0bc4f82a147e2b473adb68c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:56.891073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-be27bb32-323a-416e-9c8f-56e2313273cc None None] MFA Rules not processed for user `23f67753ed624f4a88930d55a48f2d81`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:56.896840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-be27bb32-323a-416e-9c8f-56e2313273cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:56.930345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6c434e8a-7e22-4d5a-854b-5f8e132303dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92dde467c0bc4f82a147e2b473adb68c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:56.931298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6c434e8a-7e22-4d5a-854b-5f8e132303dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92dde467c0bc4f82a147e2b473adb68c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:56.965108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 815/1638] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:17:56 2022] DELETE /identity/v3/users/92dde467c0bc4f82a147e2b473adb68c => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:56.968252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff53e0-5d49-42fa-b3b3-93a892a1dfa4 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:56.968533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff53e0-5d49-42fa-b3b3-93a892a1dfa4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:56.968821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff53e0-5d49-42fa-b3b3-93a892a1dfa4 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:56.973652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 816/1639] 10.222.0.75 () {54 vars in 912 bytes} [Fri Sep 23 15:17:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:17:56.974959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51f965-a0bd-4c45-897b-9053bcd92541 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:56.975923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51f965-a0bd-4c45-897b-9053bcd92541 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:56.976198 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51f965-a0bd-4c45-897b-9053bcd92541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:57.027940 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3e51f965-a0bd-4c45-897b-9053bcd92541 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:57.029182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3e51f965-a0bd-4c45-897b-9053bcd92541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:57.032219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:57.081791 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:57.101101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 824/1640] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:56 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:57.102457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a51259-5d47-4857-8cc4-d52edcdae9aa None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:57.102736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a51259-5d47-4857-8cc4-d52edcdae9aa None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:57.103006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a51259-5d47-4857-8cc4-d52edcdae9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:57.148876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-37a51259-5d47-4857-8cc4-d52edcdae9aa None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:57.150164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-37a51259-5d47-4857-8cc4-d52edcdae9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:57.182651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 817/1641] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:57.182651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51f965-a0bd-4c45-897b-9053bcd92541 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:57.191912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3e51f965-a0bd-4c45-897b-9053bcd92541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:57.212989 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51f965-a0bd-4c45-897b-9053bcd92541 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:57.223786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba633420-9d65-4e11-b170-d236414df333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:57.225414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba633420-9d65-4e11-b170-d236414df333 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:57.226049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba633420-9d65-4e11-b170-d236414df333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:57.226415 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba633420-9d65-4e11-b170-d236414df333 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:57.228373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba633420-9d65-4e11-b170-d236414df333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:57.232028 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba633420-9d65-4e11-b170-d236414df333 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:57.263355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 825/1642] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:17:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:57.265994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a51259-5d47-4857-8cc4-d52edcdae9aa None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:57.268171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-37a51259-5d47-4857-8cc4-d52edcdae9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:57.289148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 818/1643] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:17:57 2022] POST /identity/v3/projects => generated 405 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:57.290172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba633420-9d65-4e11-b170-d236414df333 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:57.299710 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ba633420-9d65-4e11-b170-d236414df333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:57.405655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a51259-5d47-4857-8cc4-d52edcdae9aa None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:57.422365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba633420-9d65-4e11-b170-d236414df333 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:57.430412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d2b01-f933-4887-a2aa-ecc707d216e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:57.431341 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d2b01-f933-4887-a2aa-ecc707d216e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:57.431790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d2b01-f933-4887-a2aa-ecc707d216e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:57.432302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d2b01-f933-4887-a2aa-ecc707d216e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:57.506583 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16317851-b1b3-4adc-93b1-5d4fc98097b1 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef55537360174d89ad12fa6a7354b407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44c98da3939946e8ad894efd1e56c5a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:57.510978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16317851-b1b3-4adc-93b1-5d4fc98097b1 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:57.511486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16317851-b1b3-4adc-93b1-5d4fc98097b1 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:57.512534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16317851-b1b3-4adc-93b1-5d4fc98097b1 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:57.516815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16317851-b1b3-4adc-93b1-5d4fc98097b1 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:17:57.518320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16317851-b1b3-4adc-93b1-5d4fc98097b1 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:57.521444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16317851-b1b3-4adc-93b1-5d4fc98097b1 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:57.550519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d2b01-f933-4887-a2aa-ecc707d216e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:57.553486 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d2b01-f933-4887-a2aa-ecc707d216e8 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:57.555183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 826/1644] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:17:57.556382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16317851-b1b3-4adc-93b1-5d4fc98097b1 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:57.562088 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-16317851-b1b3-4adc-93b1-5d4fc98097b1 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:57.571690 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 819/1645] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:17:57.572738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d2b01-f933-4887-a2aa-ecc707d216e8 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:57.583807 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d2b01-f933-4887-a2aa-ecc707d216e8 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:57.589483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb69ae-6fdf-45c7-a6b1-864f9c837d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:57.589678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb69ae-6fdf-45c7-a6b1-864f9c837d96 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:57.589960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb69ae-6fdf-45c7-a6b1-864f9c837d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:57.590229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb69ae-6fdf-45c7-a6b1-864f9c837d96 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:57.591406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb69ae-6fdf-45c7-a6b1-864f9c837d96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:57.593603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb69ae-6fdf-45c7-a6b1-864f9c837d96 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:57.629257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:57.635267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 820/1646] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:17:57 2022] POST /identity/v3/users => generated 309 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:57.638127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb69ae-6fdf-45c7-a6b1-864f9c837d96 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:57.654143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb69ae-6fdf-45c7-a6b1-864f9c837d96 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:57.662498 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd15d1-1ab6-47ca-b926-dbedb0a3a431 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef55537360174d89ad12fa6a7354b407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44c98da3939946e8ad894efd1e56c5a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:57.663635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd15d1-1ab6-47ca-b926-dbedb0a3a431 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:57.664016 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd15d1-1ab6-47ca-b926-dbedb0a3a431 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:57.664386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd15d1-1ab6-47ca-b926-dbedb0a3a431 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:57.665432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefd15d1-1ab6-47ca-b926-dbedb0a3a431 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:57.668166 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefd15d1-1ab6-47ca-b926-dbedb0a3a431 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:57.687511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16317851-b1b3-4adc-93b1-5d4fc98097b1 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:57.724397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64373e-b908-48e1-9944-433872a8757f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:57.724397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64373e-b908-48e1-9944-433872a8757f service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:57.724397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64373e-b908-48e1-9944-433872a8757f service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:57.724397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64373e-b908-48e1-9944-433872a8757f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:57.748463 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 821/1647] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:17:57 2022] POST /identity/v3/projects => generated 355 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:17:57.748463 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd15d1-1ab6-47ca-b926-dbedb0a3a431 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:57.757601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:57.840766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd15d1-1ab6-47ca-b926-dbedb0a3a431 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:57.849450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab863f77-c478-41b9-9ac7-4ea0c6f5d955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:57.850218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab863f77-c478-41b9-9ac7-4ea0c6f5d955 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:57.850576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab863f77-c478-41b9-9ac7-4ea0c6f5d955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:57.851070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab863f77-c478-41b9-9ac7-4ea0c6f5d955 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:57.852210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab863f77-c478-41b9-9ac7-4ea0c6f5d955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:57.854527 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab863f77-c478-41b9-9ac7-4ea0c6f5d955 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:57.864905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64373e-b908-48e1-9944-433872a8757f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:57.866371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64373e-b908-48e1-9944-433872a8757f service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:57.868067 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 822/1648] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:57.869069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab863f77-c478-41b9-9ac7-4ea0c6f5d955 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:57.887892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 827/1649] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:17:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:17:57.895865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64373e-b908-48e1-9944-433872a8757f service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:57.923229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64373e-b908-48e1-9944-433872a8757f service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:57.933155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985c75b-04ed-4cb1-a845-f148fc66c19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:57.934179 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985c75b-04ed-4cb1-a845-f148fc66c19b admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:57.934575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985c75b-04ed-4cb1-a845-f148fc66c19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:57.934974 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985c75b-04ed-4cb1-a845-f148fc66c19b admin admin] PATH_INFO: `/v3/projects/6d0d1e294d9f490ba26ea92a9d1ce226/users/e2fcf9c4bc1443989ae3a6bf793244ad/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:57.966547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab863f77-c478-41b9-9ac7-4ea0c6f5d955 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:57.971502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985c75b-04ed-4cb1-a845-f148fc66c19b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0d1e294d9f490ba26ea92a9d1ce226, user_id=e2fcf9c4bc1443989ae3a6bf793244ad, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:57.972882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50917df9-a34f-485a-acfb-197b74ea88a4 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef55537360174d89ad12fa6a7354b407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44c98da3939946e8ad894efd1e56c5a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:57.973502 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50917df9-a34f-485a-acfb-197b74ea88a4 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:57.974753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985c75b-04ed-4cb1-a845-f148fc66c19b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:57.974879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50917df9-a34f-485a-acfb-197b74ea88a4 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:57.975191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50917df9-a34f-485a-acfb-197b74ea88a4 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:57.975990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50917df9-a34f-485a-acfb-197b74ea88a4 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:17:57.976296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50917df9-a34f-485a-acfb-197b74ea88a4 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:57.978739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50917df9-a34f-485a-acfb-197b74ea88a4 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:58.002084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 828/1650] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:57 2022] PUT /identity/v3/projects/6d0d1e294d9f490ba26ea92a9d1ce226/users/e2fcf9c4bc1443989ae3a6bf793244ad/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:58.003676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 823/1651] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:17:58.007498 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985c75b-04ed-4cb1-a845-f148fc66c19b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:58.090585 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50917df9-a34f-485a-acfb-197b74ea88a4 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:58.106784 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50917df9-a34f-485a-acfb-197b74ea88a4 tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:58.207146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985c75b-04ed-4cb1-a845-f148fc66c19b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:58.224622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112532f-db80-43b2-952c-ca74096a578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:58.226775 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112532f-db80-43b2-952c-ca74096a578b admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:58.227665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112532f-db80-43b2-952c-ca74096a578b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:58.228218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112532f-db80-43b2-952c-ca74096a578b admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:58.229423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112532f-db80-43b2-952c-ca74096a578b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:58.231502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112532f-db80-43b2-952c-ca74096a578b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:58.260457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef55537360174d89ad12fa6a7354b407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '44c98da3939946e8ad894efd1e56c5a7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:58.261160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:58.261448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:58.261719 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] PATH_INFO: `/v3/projects/efa62fdfc040495c99b50703fa1e676a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:58.263564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: Authorizing `identity:delete_project(project_id=efa62fdfc040495c99b50703fa1e676a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:58.268829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:58.274112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 829/1652] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:17:58.277547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26466e45-8d70-4ffb-8048-0ca29c813d8e None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:58.277819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26466e45-8d70-4ffb-8048-0ca29c813d8e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:58.278172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26466e45-8d70-4ffb-8048-0ca29c813d8e None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:58.288227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 830/1653] 10.222.0.75 () {54 vars in 912 bytes} [Fri Sep 23 15:17:58 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:17:58.289970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26466e45-8d70-4ffb-8048-0ca29c813d8e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:58.307510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26466e45-8d70-4ffb-8048-0ca29c813d8e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:58.314049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74173b-d981-48d6-bcf2-bd69dad33c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:58.314883 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74173b-d981-48d6-bcf2-bd69dad33c82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:58.315302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74173b-d981-48d6-bcf2-bd69dad33c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:58.315572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74173b-d981-48d6-bcf2-bd69dad33c82 admin admin] PATH_INFO: `/v3/projects/6d0d1e294d9f490ba26ea92a9d1ce226/users/e2fcf9c4bc1443989ae3a6bf793244ad/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:58.317703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74173b-d981-48d6-bcf2-bd69dad33c82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0d1e294d9f490ba26ea92a9d1ce226, user_id=e2fcf9c4bc1443989ae3a6bf793244ad, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:58.320851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74173b-d981-48d6-bcf2-bd69dad33c82 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:58.346009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 831/1654] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:17:58 2022] PUT /identity/v3/projects/6d0d1e294d9f490ba26ea92a9d1ce226/users/e2fcf9c4bc1443989ae3a6bf793244ad/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:58.352652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:58.354172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89deebb1-60fb-4e09-8c0c-7da3e4e840e0 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:58.355959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89deebb1-60fb-4e09-8c0c-7da3e4e840e0 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:58.357090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89deebb1-60fb-4e09-8c0c-7da3e4e840e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:58.377415 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efa62fdfc040495c99b50703fa1e676a', 'request_id': 'req-3f633446-798c-4505-9423-00e5c630f29c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:58.389184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efa62fdfc040495c99b50703fa1e676a', 'request_id': 'req-3f633446-798c-4505-9423-00e5c630f29c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:58.405185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efa62fdfc040495c99b50703fa1e676a', 'request_id': 'req-3f633446-798c-4505-9423-00e5c630f29c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:58.410708 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-89deebb1-60fb-4e09-8c0c-7da3e4e840e0 None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:58.412834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-89deebb1-60fb-4e09-8c0c-7da3e4e840e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:58.434395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] The token cache is being invalidate because project efa62fdfc040495c99b50703fa1e676a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:17:58.434692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:58.437551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:58.441086 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 824/1655] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:17:58 2022] DELETE /identity/v3/projects/efa62fdfc040495c99b50703fa1e676a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:58.450313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:58.576658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 832/1656] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:17:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:58.590477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:58.593732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:58.757644 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89deebb1-60fb-4e09-8c0c-7da3e4e840e0 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:58.759562 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-89deebb1-60fb-4e09-8c0c-7da3e4e840e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:58.820435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f633446-798c-4505-9423-00e5c630f29c tempest-VolumeQuotaClassesTest-1484291281 tempest-VolumeQuotaClassesTest-1484291281-project-admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:58.833556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa649a-b1c3-46c6-a7c4-5dce1c4f4e82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:58.834327 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa649a-b1c3-46c6-a7c4-5dce1c4f4e82 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:58.834991 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa649a-b1c3-46c6-a7c4-5dce1c4f4e82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:58.835264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa649a-b1c3-46c6-a7c4-5dce1c4f4e82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:58.854851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa649a-b1c3-46c6-a7c4-5dce1c4f4e82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:58.858622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa649a-b1c3-46c6-a7c4-5dce1c4f4e82 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:58.904905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 825/1657] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:17:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:17:58.937840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89deebb1-60fb-4e09-8c0c-7da3e4e840e0 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:58.949772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa649a-b1c3-46c6-a7c4-5dce1c4f4e82 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:58.951277 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:58.951850 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6caa649a-b1c3-46c6-a7c4-5dce1c4f4e82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:58.952156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:58.952524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:58.952907 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] PATH_INFO: `/v3/projects/02c0792b1529441b89b3206da430c7de` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:58.955131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c0792b1529441b89b3206da430c7de)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:58.956262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: WARNING py.warnings [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 15:17:58.956262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: warnings.warn(msg) Sep 23 15:17:58.956262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: Sep 23 15:17:58.959943 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:58.992393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa649a-b1c3-46c6-a7c4-5dce1c4f4e82 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:59.017575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:59.022527 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:59.025118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:59.025471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] PATH_INFO: `/v3/users/28afc40bcc574533862417e739e366b2` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:59.027511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28afc40bcc574533862417e739e366b2)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:59.062269 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:59.097601 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c0792b1529441b89b3206da430c7de', 'request_id': 'req-627cdf87-a6e1-410e-882e-ac1ba9907485'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.121141 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c0792b1529441b89b3206da430c7de', 'request_id': 'req-627cdf87-a6e1-410e-882e-ac1ba9907485'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.137314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c0792b1529441b89b3206da430c7de', 'request_id': 'req-627cdf87-a6e1-410e-882e-ac1ba9907485'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.172330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] The token cache is being invalidate because project 02c0792b1529441b89b3206da430c7de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:17:59.172330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.172330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.176100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 833/1658] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:17:58 2022] DELETE /identity/v3/projects/02c0792b1529441b89b3206da430c7de => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:59.176853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:59.185443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:59.193769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28afc40bcc574533862417e739e366b2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.195224 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28afc40bcc574533862417e739e366b2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.225415 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28afc40bcc574533862417e739e366b2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.251393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28afc40bcc574533862417e739e366b2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.252790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28afc40bcc574533862417e739e366b2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.294594 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 826/1659] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:17:58 2022] DELETE /identity/v3/users/28afc40bcc574533862417e739e366b2 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:59.295488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:59.300809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:59.319371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:59.387865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:17:59.422793 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627cdf87-a6e1-410e-882e-ac1ba9907485 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:59.455509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9c3c2e-ce17-4478-906a-d51266da42af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:59.456305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c3c2e-ce17-4478-906a-d51266da42af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:59.456660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c3c2e-ce17-4478-906a-d51266da42af admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:59.458318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c3c2e-ce17-4478-906a-d51266da42af admin admin] PATH_INFO: `/v3/users/811321b5291b4b8f97f0a047805a1d8d` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:59.461637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87218ff2-b3ae-43b9-ac89-94ed066992b8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:17:59.462996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c3c2e-ce17-4478-906a-d51266da42af admin admin] RBAC: Authorizing `identity:delete_user(user_id=811321b5291b4b8f97f0a047805a1d8d)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:59.465187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9c3c2e-ce17-4478-906a-d51266da42af admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:59.479764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8347462-c413-40d3-a773-eba3f2db2a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:17:59.481976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8347462-c413-40d3-a773-eba3f2db2a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:59.482311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8347462-c413-40d3-a773-eba3f2db2a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:59.482583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8347462-c413-40d3-a773-eba3f2db2a2f admin admin] PATH_INFO: `/v3/users/baedced839a6484cb9ac702bb2055c59` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:59.484121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8347462-c413-40d3-a773-eba3f2db2a2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=baedced839a6484cb9ac702bb2055c59)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:17:59.486597 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8347462-c413-40d3-a773-eba3f2db2a2f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:17:59.587016 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6c9c3c2e-ce17-4478-906a-d51266da42af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '811321b5291b4b8f97f0a047805a1d8d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.594795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e8347462-c413-40d3-a773-eba3f2db2a2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baedced839a6484cb9ac702bb2055c59'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.596394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e8347462-c413-40d3-a773-eba3f2db2a2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baedced839a6484cb9ac702bb2055c59'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.602507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6c9c3c2e-ce17-4478-906a-d51266da42af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '811321b5291b4b8f97f0a047805a1d8d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.613229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e8347462-c413-40d3-a773-eba3f2db2a2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baedced839a6484cb9ac702bb2055c59'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.635199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6c9c3c2e-ce17-4478-906a-d51266da42af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '811321b5291b4b8f97f0a047805a1d8d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.649590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6c9c3c2e-ce17-4478-906a-d51266da42af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '811321b5291b4b8f97f0a047805a1d8d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.650297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6c9c3c2e-ce17-4478-906a-d51266da42af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '811321b5291b4b8f97f0a047805a1d8d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.657366 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e8347462-c413-40d3-a773-eba3f2db2a2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baedced839a6484cb9ac702bb2055c59'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.658150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 834/1660] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:17:59 2022] DELETE /identity/v3/users/811321b5291b4b8f97f0a047805a1d8d => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:59.659171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e8347462-c413-40d3-a773-eba3f2db2a2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baedced839a6484cb9ac702bb2055c59'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:17:59.659613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f79a3-ee9a-4a7c-a7ad-b2feeec7a15c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:59.659895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f79a3-ee9a-4a7c-a7ad-b2feeec7a15c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:59.660174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f79a3-ee9a-4a7c-a7ad-b2feeec7a15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:59.703840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-177f79a3-ee9a-4a7c-a7ad-b2feeec7a15c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:59.705150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-177f79a3-ee9a-4a7c-a7ad-b2feeec7a15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:59.712023 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 827/1661] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:17:59 2022] DELETE /identity/v3/users/baedced839a6484cb9ac702bb2055c59 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:17:59.715531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddd44f-813b-4fa4-94c4-4542b9d5e8c4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:17:59.716062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddd44f-813b-4fa4-94c4-4542b9d5e8c4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:17:59.716694 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ddd44f-813b-4fa4-94c4-4542b9d5e8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:17:59.789591 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-89ddd44f-813b-4fa4-94c4-4542b9d5e8c4 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:17:59.793917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-89ddd44f-813b-4fa4-94c4-4542b9d5e8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:17:59.834146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 835/1662] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:59.936763 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 828/1663] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:17:59.967219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f79a3-ee9a-4a7c-a7ad-b2feeec7a15c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:17:59.978168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-177f79a3-ee9a-4a7c-a7ad-b2feeec7a15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:00.009647 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddd44f-813b-4fa4-94c4-4542b9d5e8c4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:00.011801 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-89ddd44f-813b-4fa4-94c4-4542b9d5e8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:00.115655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f79a3-ee9a-4a7c-a7ad-b2feeec7a15c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:00.121301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd1e78-6d34-4f47-9252-e180b0660112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:00.122685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd1e78-6d34-4f47-9252-e180b0660112 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:00.123062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd1e78-6d34-4f47-9252-e180b0660112 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:00.123746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd1e78-6d34-4f47-9252-e180b0660112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:00.153857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd1e78-6d34-4f47-9252-e180b0660112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:00.156220 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdd1e78-6d34-4f47-9252-e180b0660112 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:00.178376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 836/1664] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:17:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:00.183233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ddd44f-813b-4fa4-94c4-4542b9d5e8c4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:00.188705 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26a44f-027d-4dd0-8851-5a78c9126559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:00.189590 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26a44f-027d-4dd0-8851-5a78c9126559 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:00.189901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26a44f-027d-4dd0-8851-5a78c9126559 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:00.190246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26a44f-027d-4dd0-8851-5a78c9126559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:00.202614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26a44f-027d-4dd0-8851-5a78c9126559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:00.204300 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26a44f-027d-4dd0-8851-5a78c9126559 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:00.212757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:00.224969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 829/1665] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:18:00.996154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd1e78-6d34-4f47-9252-e180b0660112 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:00.999626 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1bdd1e78-6d34-4f47-9252-e180b0660112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:01.045641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd1e78-6d34-4f47-9252-e180b0660112 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:01.069154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:01.069154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:01.069154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:01.069154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] PATH_INFO: `/v3/projects/6d6c79119ed741b88ca185ca2af3fa61` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:01.071335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d6c79119ed741b88ca185ca2af3fa61)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:01.083536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:01.085701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:01.230339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d6c79119ed741b88ca185ca2af3fa61', 'request_id': 'req-aad34921-68b4-4448-99e9-49d568ed0c1a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:01.247589 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d6c79119ed741b88ca185ca2af3fa61', 'request_id': 'req-aad34921-68b4-4448-99e9-49d568ed0c1a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:01.260550 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d6c79119ed741b88ca185ca2af3fa61', 'request_id': 'req-aad34921-68b4-4448-99e9-49d568ed0c1a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:01.302961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] The token cache is being invalidate because project 6d6c79119ed741b88ca185ca2af3fa61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:01.303612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:01.304507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:01.308852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 837/1666] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:00 2022] DELETE /identity/v3/projects/6d6c79119ed741b88ca185ca2af3fa61 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:01.315014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77eb6-4ac0-460e-889c-a50b6a4dd9ec None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:01.316507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77eb6-4ac0-460e-889c-a50b6a4dd9ec None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:01.317113 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77eb6-4ac0-460e-889c-a50b6a4dd9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:01.367822 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:01.369535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:01.380521 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-22c77eb6-4ac0-460e-889c-a50b6a4dd9ec None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:01.381804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-22c77eb6-4ac0-460e-889c-a50b6a4dd9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:01.542529 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 830/1667] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:01.545176 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857e216-81ab-4810-acfe-61df2a42a45c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:01.545846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857e216-81ab-4810-acfe-61df2a42a45c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:01.546245 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857e216-81ab-4810-acfe-61df2a42a45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:01.552290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad34921-68b4-4448-99e9-49d568ed0c1a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:01.559771 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:01.560924 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:01.561214 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:01.561816 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a admin admin] PATH_INFO: `/v3/projects/f3f04aa2dcf84d32bcf7c8cebf64feee` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:01.562670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3f04aa2dcf84d32bcf7c8cebf64feee)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:01.565548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:01.576867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a857e216-81ab-4810-acfe-61df2a42a45c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:01.578302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a857e216-81ab-4810-acfe-61df2a42a45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:01.607739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 831/1668] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:01.612575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857e216-81ab-4810-acfe-61df2a42a45c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:01.632373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857e216-81ab-4810-acfe-61df2a42a45c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:01.658518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3f04aa2dcf84d32bcf7c8cebf64feee', 'request_id': 'req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:01.671041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3f04aa2dcf84d32bcf7c8cebf64feee', 'request_id': 'req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:01.686312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3f04aa2dcf84d32bcf7c8cebf64feee', 'request_id': 'req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:01.715136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a admin admin] The token cache is being invalidate because project f3f04aa2dcf84d32bcf7c8cebf64feee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:01.715428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:01.716090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-cf3e19e4-f76d-44a0-a7da-1190d22adb2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:01.717644 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 838/1669] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:01 2022] DELETE /identity/v3/projects/f3f04aa2dcf84d32bcf7c8cebf64feee => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:18:01.722490 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2df8ae-8143-4808-a1f1-7d413aef6bad None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:01.722763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2df8ae-8143-4808-a1f1-7d413aef6bad None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:01.723019 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2df8ae-8143-4808-a1f1-7d413aef6bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:01.785615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ee2df8ae-8143-4808-a1f1-7d413aef6bad None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:01.787845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e737952-60a3-4bb5-831d-cb9d8a33f346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:01.788129 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e737952-60a3-4bb5-831d-cb9d8a33f346 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:01.789082 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e737952-60a3-4bb5-831d-cb9d8a33f346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:01.789082 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e737952-60a3-4bb5-831d-cb9d8a33f346 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:01.789384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e737952-60a3-4bb5-831d-cb9d8a33f346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:18:01.789617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e737952-60a3-4bb5-831d-cb9d8a33f346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:01.793204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ee2df8ae-8143-4808-a1f1-7d413aef6bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:01.795308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e737952-60a3-4bb5-831d-cb9d8a33f346 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:01.815714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 839/1670] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:18:01.823648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59a5cd-6e92-470f-9a24-929b6084058b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:01.823985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59a5cd-6e92-470f-9a24-929b6084058b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:01.824314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59a5cd-6e92-470f-9a24-929b6084058b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:01.826532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 832/1671] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:18:01.831336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e737952-60a3-4bb5-831d-cb9d8a33f346 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:01.834642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8e737952-60a3-4bb5-831d-cb9d8a33f346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:01.848029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1a59a5cd-6e92-470f-9a24-929b6084058b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:01.850092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1a59a5cd-6e92-470f-9a24-929b6084058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:01.861173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e737952-60a3-4bb5-831d-cb9d8a33f346 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:01.865355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 840/1672] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:18:01.872985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59a5cd-6e92-470f-9a24-929b6084058b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:01.894604 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1eea8-ec2f-4659-9299-da729f6b5b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:01.897906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59a5cd-6e92-470f-9a24-929b6084058b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:01.899184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1eea8-ec2f-4659-9299-da729f6b5b6d admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:01.899700 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1eea8-ec2f-4659-9299-da729f6b5b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:01.901246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1eea8-ec2f-4659-9299-da729f6b5b6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:01.903973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1eea8-ec2f-4659-9299-da729f6b5b6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:01.909035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1eea8-ec2f-4659-9299-da729f6b5b6d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:01.912823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab585f4b-2ed1-4ce6-87d6-e076edd52941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:01.913668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab585f4b-2ed1-4ce6-87d6-e076edd52941 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:01.915366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab585f4b-2ed1-4ce6-87d6-e076edd52941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:01.915767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab585f4b-2ed1-4ce6-87d6-e076edd52941 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:01.916554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab585f4b-2ed1-4ce6-87d6-e076edd52941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:18:01.918652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab585f4b-2ed1-4ce6-87d6-e076edd52941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:01.926123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab585f4b-2ed1-4ce6-87d6-e076edd52941 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:01.932726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:01.958444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 841/1673] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:18:01.962991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab585f4b-2ed1-4ce6-87d6-e076edd52941 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:01.993148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 833/1674] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:01 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:01.996907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1eea8-ec2f-4659-9299-da729f6b5b6d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:01.998794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab585f4b-2ed1-4ce6-87d6-e076edd52941 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:02.173839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1eea8-ec2f-4659-9299-da729f6b5b6d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:02.190525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df42152-13f9-4199-b539-ec9f3c3fa1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:02.191284 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df42152-13f9-4199-b539-ec9f3c3fa1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:02.191629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df42152-13f9-4199-b539-ec9f3c3fa1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:02.191893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df42152-13f9-4199-b539-ec9f3c3fa1c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:02.192665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df42152-13f9-4199-b539-ec9f3c3fa1c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:02.194622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df42152-13f9-4199-b539-ec9f3c3fa1c0 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:02.225162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:02.232627 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c14d58-0898-4179-8933-1542bc93624d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:02.233561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c14d58-0898-4179-8933-1542bc93624d admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:02.233948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c14d58-0898-4179-8933-1542bc93624d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:02.234364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c14d58-0898-4179-8933-1542bc93624d admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:02.235828 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c14d58-0898-4179-8933-1542bc93624d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:02.240391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c14d58-0898-4179-8933-1542bc93624d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:02.247819 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 834/1675] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:01 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:18:02.253260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df42152-13f9-4199-b539-ec9f3c3fa1c0 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:02.275417 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df42152-13f9-4199-b539-ec9f3c3fa1c0 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:02.287672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe50af-1c30-4d02-99fd-f9a26c50fa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:02.288651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe50af-1c30-4d02-99fd-f9a26c50fa9d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:02.289480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe50af-1c30-4d02-99fd-f9a26c50fa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:02.289902 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe50af-1c30-4d02-99fd-f9a26c50fa9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:02.290754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe50af-1c30-4d02-99fd-f9a26c50fa9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:02.293293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe50af-1c30-4d02-99fd-f9a26c50fa9d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:02.321447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 842/1676] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:01 2022] POST /identity/v3/projects => generated 371 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:18:02.322425 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c14d58-0898-4179-8933-1542bc93624d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:02.345377 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 835/1677] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:02.346589 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe50af-1c30-4d02-99fd-f9a26c50fa9d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:02.373358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe50af-1c30-4d02-99fd-f9a26c50fa9d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:02.472242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c14d58-0898-4179-8933-1542bc93624d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:02.478614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b831fda-2cac-46a9-8b7f-d7e667c54b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:02.479559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b831fda-2cac-46a9-8b7f-d7e667c54b2d admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:02.479902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b831fda-2cac-46a9-8b7f-d7e667c54b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:02.480276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b831fda-2cac-46a9-8b7f-d7e667c54b2d admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:02.481092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b831fda-2cac-46a9-8b7f-d7e667c54b2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:02.483342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b831fda-2cac-46a9-8b7f-d7e667c54b2d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:02.484806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5588f-1869-46b3-8aa1-2eb9605f78dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:02.486072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5588f-1869-46b3-8aa1-2eb9605f78dc admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:02.486072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5588f-1869-46b3-8aa1-2eb9605f78dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:02.486072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5588f-1869-46b3-8aa1-2eb9605f78dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:02.487026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5588f-1869-46b3-8aa1-2eb9605f78dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:02.489051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5588f-1869-46b3-8aa1-2eb9605f78dc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:02.509517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 836/1678] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:18:02.511662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5588f-1869-46b3-8aa1-2eb9605f78dc admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:02.533330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 843/1679] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:02 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:02.535862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b831fda-2cac-46a9-8b7f-d7e667c54b2d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:02.542715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5588f-1869-46b3-8aa1-2eb9605f78dc admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:02.552707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbde9b7-87af-4f3a-9d2d-e7448d703208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:02.553882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbde9b7-87af-4f3a-9d2d-e7448d703208 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:02.554370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbde9b7-87af-4f3a-9d2d-e7448d703208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:02.554813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbde9b7-87af-4f3a-9d2d-e7448d703208 admin admin] PATH_INFO: `/v3/projects/31afe21d75ba4238bf50a2f6a4e1b948/users/19aff7eae5934fd6952c14f556a45104/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:02.555472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b831fda-2cac-46a9-8b7f-d7e667c54b2d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:02.562191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89131599-b91f-4797-b2a7-ee8f1e162cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:02.571439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131599-b91f-4797-b2a7-ee8f1e162cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:02.571882 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131599-b91f-4797-b2a7-ee8f1e162cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:02.572288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131599-b91f-4797-b2a7-ee8f1e162cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:02.573400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89131599-b91f-4797-b2a7-ee8f1e162cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:02.575346 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89131599-b91f-4797-b2a7-ee8f1e162cf4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:02.580767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbde9b7-87af-4f3a-9d2d-e7448d703208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31afe21d75ba4238bf50a2f6a4e1b948, user_id=19aff7eae5934fd6952c14f556a45104, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:02.585068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbde9b7-87af-4f3a-9d2d-e7448d703208 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:02.591070 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 844/1680] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:02.595340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89131599-b91f-4797-b2a7-ee8f1e162cf4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:02.608242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89131599-b91f-4797-b2a7-ee8f1e162cf4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:02.628861 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 837/1681] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:02 2022] PUT /identity/v3/projects/31afe21d75ba4238bf50a2f6a4e1b948/users/19aff7eae5934fd6952c14f556a45104/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:18:02.632452 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c40b6-bb95-423a-b62e-805c7d1483dc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:02.632805 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c40b6-bb95-423a-b62e-805c7d1483dc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:02.633631 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c40b6-bb95-423a-b62e-805c7d1483dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:02.718108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:02.719131 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-090c40b6-bb95-423a-b62e-805c7d1483dc None None] MFA Rules not processed for user `19aff7eae5934fd6952c14f556a45104`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:02.723274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-090c40b6-bb95-423a-b62e-805c7d1483dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:02.733136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14268b91-bf92-49af-89a5-1842629e27db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:02.733761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14268b91-bf92-49af-89a5-1842629e27db admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:02.735347 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14268b91-bf92-49af-89a5-1842629e27db admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:02.735470 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14268b91-bf92-49af-89a5-1842629e27db admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:02.736145 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14268b91-bf92-49af-89a5-1842629e27db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:02.739297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14268b91-bf92-49af-89a5-1842629e27db admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:02.764312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 845/1682] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:18:02.765031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14268b91-bf92-49af-89a5-1842629e27db admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:02.773727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-14268b91-bf92-49af-89a5-1842629e27db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:02.799092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14268b91-bf92-49af-89a5-1842629e27db admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:02.815538 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:02.816084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:02.816358 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:02.816645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] PATH_INFO: `/v3/projects/dca8c512d6184ee2a57d7ca823d337e8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:02.820056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=dca8c512d6184ee2a57d7ca823d337e8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:02.822915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:02.886269 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dca8c512d6184ee2a57d7ca823d337e8', 'request_id': 'req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:02.886999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 838/1683] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:02.888008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c40b6-bb95-423a-b62e-805c7d1483dc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:02.898296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dca8c512d6184ee2a57d7ca823d337e8', 'request_id': 'req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:02.906258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dca8c512d6184ee2a57d7ca823d337e8', 'request_id': 'req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:02.907747 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c40b6-bb95-423a-b62e-805c7d1483dc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:02.922535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] The token cache is being invalidate because project dca8c512d6184ee2a57d7ca823d337e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:02.922829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:02.923480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:02.925738 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 846/1684] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:02 2022] DELETE /identity/v3/projects/dca8c512d6184ee2a57d7ca823d337e8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:02.929591 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:02.931559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:03.011228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:03.029581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fd6d7-fdd7-4b32-8b4f-0c9c2938dbcf admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:03.031452 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214f9d6-d6ed-444b-b08b-17add6860989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:03.032305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214f9d6-d6ed-444b-b08b-17add6860989 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:03.032754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214f9d6-d6ed-444b-b08b-17add6860989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:03.033195 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214f9d6-d6ed-444b-b08b-17add6860989 admin admin] PATH_INFO: `/v3/projects/33a269c585ec4ba6b81d904d3395fb8d/users/a0d6672e30f1417fa386df6925e375f6/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:03.036527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953095e0-081d-4000-ab2f-f89413cdc335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:03.037464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953095e0-081d-4000-ab2f-f89413cdc335 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:03.037754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953095e0-081d-4000-ab2f-f89413cdc335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:03.038184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953095e0-081d-4000-ab2f-f89413cdc335 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:03.039012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953095e0-081d-4000-ab2f-f89413cdc335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:03.044424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953095e0-081d-4000-ab2f-f89413cdc335 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:03.073650 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214f9d6-d6ed-444b-b08b-17add6860989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a269c585ec4ba6b81d904d3395fb8d, user_id=a0d6672e30f1417fa386df6925e375f6, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:03.077144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214f9d6-d6ed-444b-b08b-17add6860989 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:03.087014 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 847/1685] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:02 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:03.088421 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953095e0-081d-4000-ab2f-f89413cdc335 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:03.092234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-953095e0-081d-4000-ab2f-f89413cdc335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:03.109436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 839/1686] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:02 2022] PUT /identity/v3/projects/33a269c585ec4ba6b81d904d3395fb8d/users/a0d6672e30f1417fa386df6925e375f6/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:03.110468 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214f9d6-d6ed-444b-b08b-17add6860989 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:03.232980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953095e0-081d-4000-ab2f-f89413cdc335 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:03.239719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3627871-9738-46d8-9d96-56a6cb922ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:03.241165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3627871-9738-46d8-9d96-56a6cb922ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:03.241509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3627871-9738-46d8-9d96-56a6cb922ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:03.242132 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3627871-9738-46d8-9d96-56a6cb922ad5 admin admin] PATH_INFO: `/v3/users/ef55537360174d89ad12fa6a7354b407` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:03.243452 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3627871-9738-46d8-9d96-56a6cb922ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef55537360174d89ad12fa6a7354b407)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:03.248083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3627871-9738-46d8-9d96-56a6cb922ad5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:03.249528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214f9d6-d6ed-444b-b08b-17add6860989 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:03.251777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:03.265469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb55dc-ed12-444d-ad6f-7dd3d92b2e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:03.266425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb55dc-ed12-444d-ad6f-7dd3d92b2e96 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:03.266538 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb55dc-ed12-444d-ad6f-7dd3d92b2e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:03.266809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb55dc-ed12-444d-ad6f-7dd3d92b2e96 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:03.268480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb55dc-ed12-444d-ad6f-7dd3d92b2e96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:03.274084 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb55dc-ed12-444d-ad6f-7dd3d92b2e96 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:03.336576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 840/1687] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:03 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:03.338925 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67a6ef-4a3d-4c34-a457-c2eb45d1c1bb None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:03.339047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67a6ef-4a3d-4c34-a457-c2eb45d1c1bb None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:03.339604 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67a6ef-4a3d-4c34-a457-c2eb45d1c1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:03.355592 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c3627871-9738-46d8-9d96-56a6cb922ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef55537360174d89ad12fa6a7354b407'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:03.376958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c3627871-9738-46d8-9d96-56a6cb922ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef55537360174d89ad12fa6a7354b407'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:03.393041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0b67a6ef-4a3d-4c34-a457-c2eb45d1c1bb None None] MFA Rules not processed for user `a0d6672e30f1417fa386df6925e375f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:03.394433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0b67a6ef-4a3d-4c34-a457-c2eb45d1c1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:03.400790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c3627871-9738-46d8-9d96-56a6cb922ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef55537360174d89ad12fa6a7354b407'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:03.420171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c3627871-9738-46d8-9d96-56a6cb922ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef55537360174d89ad12fa6a7354b407'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:03.421024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c3627871-9738-46d8-9d96-56a6cb922ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef55537360174d89ad12fa6a7354b407'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:03.434309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 848/1688] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:03 2022] DELETE /identity/v3/users/ef55537360174d89ad12fa6a7354b407 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:03.435672 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f78cb-6584-4f11-86f4-e65115559ba0 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:03.435959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f78cb-6584-4f11-86f4-e65115559ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:03.438439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f78cb-6584-4f11-86f4-e65115559ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:03.486902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-498f78cb-6584-4f11-86f4-e65115559ba0 None None] MFA Rules not processed for user `e2fcf9c4bc1443989ae3a6bf793244ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:03.488703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-498f78cb-6584-4f11-86f4-e65115559ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:03.546429 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 841/1689] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:03.547386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67a6ef-4a3d-4c34-a457-c2eb45d1c1bb None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:03.551920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0b67a6ef-4a3d-4c34-a457-c2eb45d1c1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:03.581511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67a6ef-4a3d-4c34-a457-c2eb45d1c1bb None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:03.667591 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 849/1690] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:03 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:03.669611 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f78cb-6584-4f11-86f4-e65115559ba0 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:03.676202 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-498f78cb-6584-4f11-86f4-e65115559ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:03.714089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:03.740850 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2085e-969d-4e5a-a77c-57073871211c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:03.741686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2085e-969d-4e5a-a77c-57073871211c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:03.742003 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2085e-969d-4e5a-a77c-57073871211c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:03.742276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2085e-969d-4e5a-a77c-57073871211c admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:03.744378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2085e-969d-4e5a-a77c-57073871211c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:03.748441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2085e-969d-4e5a-a77c-57073871211c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:03.778285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 842/1691] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:03.784137 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2085e-969d-4e5a-a77c-57073871211c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:03.805077 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2085e-969d-4e5a-a77c-57073871211c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:03.816373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51752b2a-9b1f-40b5-9b89-057467af10c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:03.817092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752b2a-9b1f-40b5-9b89-057467af10c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:03.817390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752b2a-9b1f-40b5-9b89-057467af10c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:03.818404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51752b2a-9b1f-40b5-9b89-057467af10c8 admin admin] PATH_INFO: `/v3/projects/c6a818815c7c404eb86bb41e1cde04b7/users/18680d388391427db7e3e1d95774004d/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:03.824846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f78cb-6584-4f11-86f4-e65115559ba0 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:03.838499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffb394-faeb-4508-ac87-710095f355ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:03.839236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffb394-faeb-4508-ac87-710095f355ed service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:03.839510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffb394-faeb-4508-ac87-710095f355ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:03.839780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffb394-faeb-4508-ac87-710095f355ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:03.868846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51752b2a-9b1f-40b5-9b89-057467af10c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a818815c7c404eb86bb41e1cde04b7, user_id=18680d388391427db7e3e1d95774004d, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:03.878830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51752b2a-9b1f-40b5-9b89-057467af10c8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:03.897076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:03.913128 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 843/1692] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:03 2022] PUT /identity/v3/projects/c6a818815c7c404eb86bb41e1cde04b7/users/18680d388391427db7e3e1d95774004d/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:03.914370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51752b2a-9b1f-40b5-9b89-057467af10c8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:04.006660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffb394-faeb-4508-ac87-710095f355ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:04.010695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffb394-faeb-4508-ac87-710095f355ed service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:04.033278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 850/1693] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:04.060193 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffb394-faeb-4508-ac87-710095f355ed service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:04.061995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-66ffb394-faeb-4508-ac87-710095f355ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:04.104914 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51752b2a-9b1f-40b5-9b89-057467af10c8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:04.119134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffb394-faeb-4508-ac87-710095f355ed service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:04.125313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389c94c-a0c1-4d9f-a9c6-2a29bab2a52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:04.127703 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389c94c-a0c1-4d9f-a9c6-2a29bab2a52f admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:04.129181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389c94c-a0c1-4d9f-a9c6-2a29bab2a52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:04.129566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389c94c-a0c1-4d9f-a9c6-2a29bab2a52f admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:04.130396 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043256be-7808-49e3-a687-ccd624325f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:04.131235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389c94c-a0c1-4d9f-a9c6-2a29bab2a52f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:04.131988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043256be-7808-49e3-a687-ccd624325f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:04.133806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389c94c-a0c1-4d9f-a9c6-2a29bab2a52f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:04.134229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043256be-7808-49e3-a687-ccd624325f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:04.134602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043256be-7808-49e3-a687-ccd624325f18 admin admin] PATH_INFO: `/v3/users/a0d6672e30f1417fa386df6925e375f6` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:04.137087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043256be-7808-49e3-a687-ccd624325f18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d6672e30f1417fa386df6925e375f6)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:04.142069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043256be-7808-49e3-a687-ccd624325f18 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:04.163919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 844/1694] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:04.166056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389c94c-a0c1-4d9f-a9c6-2a29bab2a52f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:04.194257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389c94c-a0c1-4d9f-a9c6-2a29bab2a52f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:04.205112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a39f77-8b80-4752-b9c3-dccaca2f7c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:04.205514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a39f77-8b80-4752-b9c3-dccaca2f7c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:04.205796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a39f77-8b80-4752-b9c3-dccaca2f7c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:04.206949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a39f77-8b80-4752-b9c3-dccaca2f7c0b admin admin] PATH_INFO: `/v3/projects/c6a818815c7c404eb86bb41e1cde04b7/users/18680d388391427db7e3e1d95774004d/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:04.209349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a39f77-8b80-4752-b9c3-dccaca2f7c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a818815c7c404eb86bb41e1cde04b7, user_id=18680d388391427db7e3e1d95774004d, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:04.212727 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a39f77-8b80-4752-b9c3-dccaca2f7c0b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:04.262616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 845/1695] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:04 2022] PUT /identity/v3/projects/c6a818815c7c404eb86bb41e1cde04b7/users/18680d388391427db7e3e1d95774004d/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:04.264337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a83d32-d0d4-4006-9497-bfb87007201d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:04.265461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a83d32-d0d4-4006-9497-bfb87007201d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:04.265738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a83d32-d0d4-4006-9497-bfb87007201d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:04.270368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-043256be-7808-49e3-a687-ccd624325f18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d6672e30f1417fa386df6925e375f6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:04.289971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-043256be-7808-49e3-a687-ccd624325f18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d6672e30f1417fa386df6925e375f6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:04.324756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-043256be-7808-49e3-a687-ccd624325f18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d6672e30f1417fa386df6925e375f6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:04.340825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c2a83d32-d0d4-4006-9497-bfb87007201d None None] MFA Rules not processed for user `18680d388391427db7e3e1d95774004d`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:04.342138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c2a83d32-d0d4-4006-9497-bfb87007201d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:04.345690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-043256be-7808-49e3-a687-ccd624325f18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d6672e30f1417fa386df6925e375f6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:04.346387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-043256be-7808-49e3-a687-ccd624325f18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d6672e30f1417fa386df6925e375f6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:04.365962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 851/1696] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:04 2022] DELETE /identity/v3/users/a0d6672e30f1417fa386df6925e375f6 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:04.369362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f232cb-522d-4335-b970-3b53d28e6525 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:04.369646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f232cb-522d-4335-b970-3b53d28e6525 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:04.371051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f232cb-522d-4335-b970-3b53d28e6525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:04.417964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-87f232cb-522d-4335-b970-3b53d28e6525 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:04.419225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-87f232cb-522d-4335-b970-3b53d28e6525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:04.536704 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 846/1697] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:04.545061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a83d32-d0d4-4006-9497-bfb87007201d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:04.546435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c2a83d32-d0d4-4006-9497-bfb87007201d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:04.575831 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 852/1698] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:04.610132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:04.660486 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a83d32-d0d4-4006-9497-bfb87007201d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:04.688989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f232cb-522d-4335-b970-3b53d28e6525 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:04.690826 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-87f232cb-522d-4335-b970-3b53d28e6525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:04.756412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e49a6-6ca0-455c-b89f-f864b4f26f6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:04.757418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e49a6-6ca0-455c-b89f-f864b4f26f6d service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:04.757865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e49a6-6ca0-455c-b89f-f864b4f26f6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:04.758299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e49a6-6ca0-455c-b89f-f864b4f26f6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:04.761567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-476e49a6-6ca0-455c-b89f-f864b4f26f6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:04.785571 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:04.819354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f232cb-522d-4335-b970-3b53d28e6525 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:04.826420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54c39b-bba6-493f-8f95-db2a7a0a9594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:04.827333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54c39b-bba6-493f-8f95-db2a7a0a9594 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:04.827823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54c39b-bba6-493f-8f95-db2a7a0a9594 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:04.828160 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54c39b-bba6-493f-8f95-db2a7a0a9594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:04.839388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54c39b-bba6-493f-8f95-db2a7a0a9594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:04.841024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54c39b-bba6-493f-8f95-db2a7a0a9594 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:04.854280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 853/1699] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:04.875852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e49a6-6ca0-455c-b89f-f864b4f26f6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:04.878371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e49a6-6ca0-455c-b89f-f864b4f26f6d service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:04.901821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 847/1700] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:05.532282 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54c39b-bba6-493f-8f95-db2a7a0a9594 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:05.534691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ce54c39b-bba6-493f-8f95-db2a7a0a9594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:05.567366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54c39b-bba6-493f-8f95-db2a7a0a9594 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:05.586724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc89168-f61a-44f7-b8b7-833fac9728ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:05.587384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc89168-f61a-44f7-b8b7-833fac9728ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:05.587655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc89168-f61a-44f7-b8b7-833fac9728ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:05.587921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc89168-f61a-44f7-b8b7-833fac9728ba admin admin] PATH_INFO: `/v3/projects/44c98da3939946e8ad894efd1e56c5a7` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:05.592814 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc89168-f61a-44f7-b8b7-833fac9728ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c98da3939946e8ad894efd1e56c5a7)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:05.594754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc89168-f61a-44f7-b8b7-833fac9728ba admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:05.735380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-efc89168-f61a-44f7-b8b7-833fac9728ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c98da3939946e8ad894efd1e56c5a7', 'request_id': 'req-efc89168-f61a-44f7-b8b7-833fac9728ba'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:05.745884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-efc89168-f61a-44f7-b8b7-833fac9728ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c98da3939946e8ad894efd1e56c5a7', 'request_id': 'req-efc89168-f61a-44f7-b8b7-833fac9728ba'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:05.754747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-efc89168-f61a-44f7-b8b7-833fac9728ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c98da3939946e8ad894efd1e56c5a7', 'request_id': 'req-efc89168-f61a-44f7-b8b7-833fac9728ba'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:05.777921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-efc89168-f61a-44f7-b8b7-833fac9728ba admin admin] The token cache is being invalidate because project 44c98da3939946e8ad894efd1e56c5a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:05.778489 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-efc89168-f61a-44f7-b8b7-833fac9728ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:05.780749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-efc89168-f61a-44f7-b8b7-833fac9728ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:05.788907 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 854/1701] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:05 2022] DELETE /identity/v3/projects/44c98da3939946e8ad894efd1e56c5a7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:05.791515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75ea20-4ab0-4001-9669-9d21031f6d19 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:05.791962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75ea20-4ab0-4001-9669-9d21031f6d19 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:05.792325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75ea20-4ab0-4001-9669-9d21031f6d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:05.901947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-fd75ea20-4ab0-4001-9669-9d21031f6d19 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:05.906093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fd75ea20-4ab0-4001-9669-9d21031f6d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:06.036282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 848/1702] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:18:06.045708 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f97fbd-c15e-4d08-ad54-d5a9b78b6a1c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:06.045987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f97fbd-c15e-4d08-ad54-d5a9b78b6a1c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:06.046263 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f97fbd-c15e-4d08-ad54-d5a9b78b6a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:06.099984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-84f97fbd-c15e-4d08-ad54-d5a9b78b6a1c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:06.101360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-84f97fbd-c15e-4d08-ad54-d5a9b78b6a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:06.126505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 855/1703] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:06.131002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75ea20-4ab0-4001-9669-9d21031f6d19 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:06.144761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75ea20-4ab0-4001-9669-9d21031f6d19 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:06.159439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80a483-06b6-4ddc-b362-e1f64369bb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:06.160229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80a483-06b6-4ddc-b362-e1f64369bb7e admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:06.160610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80a483-06b6-4ddc-b362-e1f64369bb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:06.160980 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80a483-06b6-4ddc-b362-e1f64369bb7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:06.161744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80a483-06b6-4ddc-b362-e1f64369bb7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:18:06.162137 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80a483-06b6-4ddc-b362-e1f64369bb7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:06.164846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80a483-06b6-4ddc-b362-e1f64369bb7e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:06.202714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 849/1704] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:18:06.203961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f97fbd-c15e-4d08-ad54-d5a9b78b6a1c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:06.239774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f97fbd-c15e-4d08-ad54-d5a9b78b6a1c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:06.271828 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa8154-7f2b-4489-833e-d01d8447e074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:06.272510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa8154-7f2b-4489-833e-d01d8447e074 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:06.272851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa8154-7f2b-4489-833e-d01d8447e074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:06.273337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa8154-7f2b-4489-833e-d01d8447e074 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:06.274006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa8154-7f2b-4489-833e-d01d8447e074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:06.275970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa8154-7f2b-4489-833e-d01d8447e074 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:06.343067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 856/1705] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:06 2022] POST /identity/v3/projects => generated 391 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:06.348483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80a483-06b6-4ddc-b362-e1f64369bb7e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:06.366523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80a483-06b6-4ddc-b362-e1f64369bb7e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:06.413825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:06.504479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9cc7b-93c6-4ac8-b9d5-677029a43e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:06.506563 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9cc7b-93c6-4ac8-b9d5-677029a43e55 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:06.507124 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9cc7b-93c6-4ac8-b9d5-677029a43e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:06.509137 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9cc7b-93c6-4ac8-b9d5-677029a43e55 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:06.510061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9cc7b-93c6-4ac8-b9d5-677029a43e55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:06.513823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9cc7b-93c6-4ac8-b9d5-677029a43e55 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:06.559485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 850/1706] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:06 2022] POST /identity/v3/users => generated 296 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:06.560087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa8154-7f2b-4489-833e-d01d8447e074 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:06.575232 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa8154-7f2b-4489-833e-d01d8447e074 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:06.582735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f03d04-0c58-42a9-9ea2-c957b2706e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:06.583816 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f03d04-0c58-42a9-9ea2-c957b2706e57 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:06.584509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f03d04-0c58-42a9-9ea2-c957b2706e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:06.585075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f03d04-0c58-42a9-9ea2-c957b2706e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:06.585975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f03d04-0c58-42a9-9ea2-c957b2706e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:06.587903 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f03d04-0c58-42a9-9ea2-c957b2706e57 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:06.608292 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 857/1707] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:18:06.608991 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ec1e0-7196-4349-8109-0ab14d23ba07 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:06.609281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ec1e0-7196-4349-8109-0ab14d23ba07 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:06.609557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ec1e0-7196-4349-8109-0ab14d23ba07 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:06.611073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f03d04-0c58-42a9-9ea2-c957b2706e57 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:06.611523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 851/1708] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:18:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:18:06.621304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbfef9-f305-4ac3-9497-938df659ca21 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:06.621558 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbfef9-f305-4ac3-9497-938df659ca21 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:06.621794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbfef9-f305-4ac3-9497-938df659ca21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:06.629638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f03d04-0c58-42a9-9ea2-c957b2706e57 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:06.636693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b2410-add5-4454-80df-62c844925f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:06.638275 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b2410-add5-4454-80df-62c844925f6e admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:06.638798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b2410-add5-4454-80df-62c844925f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:06.639265 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b2410-add5-4454-80df-62c844925f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:06.640150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b2410-add5-4454-80df-62c844925f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:06.645995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b2410-add5-4454-80df-62c844925f6e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:06.659470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0cfbfef9-f305-4ac3-9497-938df659ca21 None None] MFA Rules not processed for user `2b8be1ec4a7a45b88d84bd9c48fab407`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:06.660771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0cfbfef9-f305-4ac3-9497-938df659ca21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:06.668316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 858/1709] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:06.669613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b2410-add5-4454-80df-62c844925f6e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:06.702451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b2410-add5-4454-80df-62c844925f6e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:06.710022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbb06f-7a4b-4561-9d95-3b66494a1898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:06.711135 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbb06f-7a4b-4561-9d95-3b66494a1898 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:06.711610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbb06f-7a4b-4561-9d95-3b66494a1898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:06.712390 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbb06f-7a4b-4561-9d95-3b66494a1898 admin admin] PATH_INFO: `/v3/projects/0f71446c610743abb96c197797ff80d1/users/9d0a6813289b421c876db35565144d8c/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:06.748841 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbb06f-7a4b-4561-9d95-3b66494a1898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f71446c610743abb96c197797ff80d1, user_id=9d0a6813289b421c876db35565144d8c, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:06.753128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fbb06f-7a4b-4561-9d95-3b66494a1898 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:06.786296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 859/1710] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:06 2022] PUT /identity/v3/projects/0f71446c610743abb96c197797ff80d1/users/9d0a6813289b421c876db35565144d8c/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:06.798525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457c4de-4911-42a8-a07a-a3b41d936722 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:06.799905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457c4de-4911-42a8-a07a-a3b41d936722 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:06.800399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457c4de-4911-42a8-a07a-a3b41d936722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:06.809678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 852/1711] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:18:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:18:06.850669 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6457c4de-4911-42a8-a07a-a3b41d936722 None None] MFA Rules not processed for user `9d0a6813289b421c876db35565144d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:06.853904 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6457c4de-4911-42a8-a07a-a3b41d936722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:06.879829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbfef9-f305-4ac3-9497-938df659ca21 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:06.881290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0cfbfef9-f305-4ac3-9497-938df659ca21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:06.903470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbfef9-f305-4ac3-9497-938df659ca21 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:06.931353 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:07.047149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 860/1712] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:07.047149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9dc9a-3d8a-429a-8982-37ac370ae849 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:07.047149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9dc9a-3d8a-429a-8982-37ac370ae849 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:07.047149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9dc9a-3d8a-429a-8982-37ac370ae849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:07.087625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74333650-92f0-4869-8b9c-2ad0b3bad64d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:07.089638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74333650-92f0-4869-8b9c-2ad0b3bad64d service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:07.090146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74333650-92f0-4869-8b9c-2ad0b3bad64d service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:07.090604 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74333650-92f0-4869-8b9c-2ad0b3bad64d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:07.103773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74333650-92f0-4869-8b9c-2ad0b3bad64d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:07.105875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74333650-92f0-4869-8b9c-2ad0b3bad64d service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:07.134628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f9b9dc9a-3d8a-429a-8982-37ac370ae849 None None] MFA Rules not processed for user `4cfac62e1c304167abb971815d5da358`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:07.138998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 853/1713] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:18:07.140299 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f9b9dc9a-3d8a-429a-8982-37ac370ae849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:07.142290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74333650-92f0-4869-8b9c-2ad0b3bad64d service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:07.356836 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:07.383063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 861/1714] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:07.401150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e12f4-7005-4f91-9d3e-bee1ae600685 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:07.402154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e12f4-7005-4f91-9d3e-bee1ae600685 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:07.402634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e12f4-7005-4f91-9d3e-bee1ae600685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:07.415497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74333650-92f0-4869-8b9c-2ad0b3bad64d service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:07.427973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401856e-5f18-4f2d-a908-e3eaea3617b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:07.428641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401856e-5f18-4f2d-a908-e3eaea3617b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:07.428923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401856e-5f18-4f2d-a908-e3eaea3617b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:07.429189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7401856e-5f18-4f2d-a908-e3eaea3617b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:07.429849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401856e-5f18-4f2d-a908-e3eaea3617b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:07.436319 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7401856e-5f18-4f2d-a908-e3eaea3617b2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:07.460559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f96e12f4-7005-4f91-9d3e-bee1ae600685 None None] MFA Rules not processed for user `4cfac62e1c304167abb971815d5da358`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:07.463684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f96e12f4-7005-4f91-9d3e-bee1ae600685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:07.478089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 854/1715] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:07 2022] POST /identity/v3/projects => generated 391 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:07.479927 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401856e-5f18-4f2d-a908-e3eaea3617b2 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:07.487046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 862/1716] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:07.504392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e12f4-7005-4f91-9d3e-bee1ae600685 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:07.505934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f96e12f4-7005-4f91-9d3e-bee1ae600685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:07.562917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7401856e-5f18-4f2d-a908-e3eaea3617b2 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:07.565163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1d44d-b949-43d3-a323-8bc9a27adce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:07.566563 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1d44d-b949-43d3-a323-8bc9a27adce1 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:07.566959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1d44d-b949-43d3-a323-8bc9a27adce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:07.567332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1d44d-b949-43d3-a323-8bc9a27adce1 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:07.568109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1d44d-b949-43d3-a323-8bc9a27adce1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:07.570409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1d44d-b949-43d3-a323-8bc9a27adce1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:07.610855 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 855/1717] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:07 2022] POST /identity/v3/users => generated 302 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:07.612935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1d44d-b949-43d3-a323-8bc9a27adce1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:07.645123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1d44d-b949-43d3-a323-8bc9a27adce1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:07.659611 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d779c-1005-4bff-beda-2a6272933ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:07.660290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d779c-1005-4bff-beda-2a6272933ded admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:07.661174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d779c-1005-4bff-beda-2a6272933ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:07.661174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d779c-1005-4bff-beda-2a6272933ded admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:07.661778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d779c-1005-4bff-beda-2a6272933ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:07.663705 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d779c-1005-4bff-beda-2a6272933ded admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:07.718793 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 856/1718] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:07.724791 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d779c-1005-4bff-beda-2a6272933ded admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:07.751955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d779c-1005-4bff-beda-2a6272933ded admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:07.767451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f399f-8c2b-442d-a223-8dfd1d9af904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:07.768952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f399f-8c2b-442d-a223-8dfd1d9af904 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:07.772298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f399f-8c2b-442d-a223-8dfd1d9af904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:07.772903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f399f-8c2b-442d-a223-8dfd1d9af904 admin admin] PATH_INFO: `/v3/projects/14a33604d47a45b8b25b87be0fc326cb/users/15f79a38d3934a9084ca04b3c8d6379a/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:07.783257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e12f4-7005-4f91-9d3e-bee1ae600685 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:07.823794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f399f-8c2b-442d-a223-8dfd1d9af904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a33604d47a45b8b25b87be0fc326cb, user_id=15f79a38d3934a9084ca04b3c8d6379a, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:07.826398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f399f-8c2b-442d-a223-8dfd1d9af904 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:07.871869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 857/1719] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:07 2022] PUT /identity/v3/projects/14a33604d47a45b8b25b87be0fc326cb/users/15f79a38d3934a9084ca04b3c8d6379a/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:07.874729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f399f-8c2b-442d-a223-8dfd1d9af904 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:07.925328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:07.983577 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58099e6d-e8c7-48ab-a0e9-5f5d014d8cda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:07.988067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58099e6d-e8c7-48ab-a0e9-5f5d014d8cda service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:07.988368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58099e6d-e8c7-48ab-a0e9-5f5d014d8cda service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:07.988465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58099e6d-e8c7-48ab-a0e9-5f5d014d8cda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:08.118339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f399f-8c2b-442d-a223-8dfd1d9af904 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:08.126325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565cec2-894b-48af-9863-efecba14b975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:08.127314 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565cec2-894b-48af-9863-efecba14b975 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:08.127589 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565cec2-894b-48af-9863-efecba14b975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:08.127860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565cec2-894b-48af-9863-efecba14b975 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:08.128549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565cec2-894b-48af-9863-efecba14b975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:08.130250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565cec2-894b-48af-9863-efecba14b975 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:08.161399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 858/1720] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:08.167866 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565cec2-894b-48af-9863-efecba14b975 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:08.189371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565cec2-894b-48af-9863-efecba14b975 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:08.193052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58099e6d-e8c7-48ab-a0e9-5f5d014d8cda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:08.194657 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58099e6d-e8c7-48ab-a0e9-5f5d014d8cda service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:08.199020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39061e3c-51e2-4448-84ea-8bc8be755baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:08.200143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39061e3c-51e2-4448-84ea-8bc8be755baf admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:08.200143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39061e3c-51e2-4448-84ea-8bc8be755baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:08.200326 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39061e3c-51e2-4448-84ea-8bc8be755baf admin admin] PATH_INFO: `/v3/projects/14a33604d47a45b8b25b87be0fc326cb/users/15f79a38d3934a9084ca04b3c8d6379a/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:08.210808 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39061e3c-51e2-4448-84ea-8bc8be755baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a33604d47a45b8b25b87be0fc326cb, user_id=15f79a38d3934a9084ca04b3c8d6379a, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:08.219112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39061e3c-51e2-4448-84ea-8bc8be755baf admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:08.223005 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 863/1721] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:18:08.253555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 859/1722] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:08 2022] PUT /identity/v3/projects/14a33604d47a45b8b25b87be0fc326cb/users/15f79a38d3934a9084ca04b3c8d6379a/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:18:08.264392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46028d1b-d490-4c91-b373-f674096510ba None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:08.264392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46028d1b-d490-4c91-b373-f674096510ba None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:08.264692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46028d1b-d490-4c91-b373-f674096510ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:08.274281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39061e3c-51e2-4448-84ea-8bc8be755baf admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:08.291861 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-39061e3c-51e2-4448-84ea-8bc8be755baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:08.314173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:08.350258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-46028d1b-d490-4c91-b373-f674096510ba None None] MFA Rules not processed for user `15f79a38d3934a9084ca04b3c8d6379a`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:08.351832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-46028d1b-d490-4c91-b373-f674096510ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:08.492449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39061e3c-51e2-4448-84ea-8bc8be755baf admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:08.537392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:08.537851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:08.538131 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:08.538397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e admin admin] PATH_INFO: `/v3/projects/33a269c585ec4ba6b81d904d3395fb8d` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:08.545052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=33a269c585ec4ba6b81d904d3395fb8d)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:08.547923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:08.639936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 864/1723] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:08 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:08.654901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46028d1b-d490-4c91-b373-f674096510ba None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:08.656344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-46028d1b-d490-4c91-b373-f674096510ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:08.721642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33a269c585ec4ba6b81d904d3395fb8d', 'request_id': 'req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:08.741928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33a269c585ec4ba6b81d904d3395fb8d', 'request_id': 'req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:08.757542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33a269c585ec4ba6b81d904d3395fb8d', 'request_id': 'req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:08.790478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e admin admin] The token cache is being invalidate because project 33a269c585ec4ba6b81d904d3395fb8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:08.791405 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:08.792456 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-012a22fa-7b9c-4df8-ae0f-6d7bdfd5ee8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:08.800165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 860/1724] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:08 2022] DELETE /identity/v3/projects/33a269c585ec4ba6b81d904d3395fb8d => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:08.801978 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b9b8c-dc67-4d21-bef4-b8c1a997d2d4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:08.802346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b9b8c-dc67-4d21-bef4-b8c1a997d2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:08.802770 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b9b8c-dc67-4d21-bef4-b8c1a997d2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:08.842395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-184b9b8c-dc67-4d21-bef4-b8c1a997d2d4 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:08.843129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46028d1b-d490-4c91-b373-f674096510ba None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:08.846083 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-184b9b8c-dc67-4d21-bef4-b8c1a997d2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:08.910741 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:08.976443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff4e6e-555e-4e2c-bb71-edebc50b2d41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:08.977155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff4e6e-555e-4e2c-bb71-edebc50b2d41 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:08.977457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff4e6e-555e-4e2c-bb71-edebc50b2d41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:08.977692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff4e6e-555e-4e2c-bb71-edebc50b2d41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:08.993677 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-54ff4e6e-555e-4e2c-bb71-edebc50b2d41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:09.033505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 861/1725] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:09.057346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b27512-5272-48b9-8a51-de6ddb8accbc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:09.058053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b27512-5272-48b9-8a51-de6ddb8accbc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:09.058531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b27512-5272-48b9-8a51-de6ddb8accbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:09.107411 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c5b27512-5272-48b9-8a51-de6ddb8accbc None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:09.108780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c5b27512-5272-48b9-8a51-de6ddb8accbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:09.122574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 862/1726] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:09.126954 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b27512-5272-48b9-8a51-de6ddb8accbc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:09.145690 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b27512-5272-48b9-8a51-de6ddb8accbc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:09.152022 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3ac34-2145-4dac-b837-ca93e0062ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:09.152880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3ac34-2145-4dac-b837-ca93e0062ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:09.153252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3ac34-2145-4dac-b837-ca93e0062ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:09.153655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3ac34-2145-4dac-b837-ca93e0062ed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:09.155105 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3ac34-2145-4dac-b837-ca93e0062ed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:18:09.155562 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3ac34-2145-4dac-b837-ca93e0062ed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:09.157403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3ac34-2145-4dac-b837-ca93e0062ed2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:09.184526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff4e6e-555e-4e2c-bb71-edebc50b2d41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:09.187614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff4e6e-555e-4e2c-bb71-edebc50b2d41 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:09.188054 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 863/1727] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:18:09.192525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3ac34-2145-4dac-b837-ca93e0062ed2 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:09.207831 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3ac34-2145-4dac-b837-ca93e0062ed2 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:09.210054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 865/1728] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:08 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:18:09.215800 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d7e51-7574-420e-a168-56a02b2723e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:09.217064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d7e51-7574-420e-a168-56a02b2723e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:09.217448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d7e51-7574-420e-a168-56a02b2723e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:09.217812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d7e51-7574-420e-a168-56a02b2723e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:09.219416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d7e51-7574-420e-a168-56a02b2723e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:09.221775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d7e51-7574-420e-a168-56a02b2723e6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:09.267120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:09.299772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 864/1729] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:09 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:09.301234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff4e6e-555e-4e2c-bb71-edebc50b2d41 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:09.319373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff4e6e-555e-4e2c-bb71-edebc50b2d41 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:09.355665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d7e51-7574-420e-a168-56a02b2723e6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:09.357834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d61d7e51-7574-420e-a168-56a02b2723e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:09.435417 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846bb68-51a4-4d24-8053-e87dfb80415a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:09.436080 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846bb68-51a4-4d24-8053-e87dfb80415a admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:09.437079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846bb68-51a4-4d24-8053-e87dfb80415a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:09.437375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846bb68-51a4-4d24-8053-e87dfb80415a admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:09.439274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846bb68-51a4-4d24-8053-e87dfb80415a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:09.442355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846bb68-51a4-4d24-8053-e87dfb80415a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:09.496621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 866/1730] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:09 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:09.497612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846bb68-51a4-4d24-8053-e87dfb80415a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:09.520043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d7e51-7574-420e-a168-56a02b2723e6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:09.526041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846bb68-51a4-4d24-8053-e87dfb80415a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:09.540873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fef6da-ebc3-4874-943c-88b6a575a1f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:09.541576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fef6da-ebc3-4874-943c-88b6a575a1f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:09.541853 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fef6da-ebc3-4874-943c-88b6a575a1f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:09.542178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fef6da-ebc3-4874-943c-88b6a575a1f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:09.543186 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c761c-3c79-451c-a4e7-bb60ec0a96e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:09.544215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c761c-3c79-451c-a4e7-bb60ec0a96e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:09.544577 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c761c-3c79-451c-a4e7-bb60ec0a96e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:09.544979 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c761c-3c79-451c-a4e7-bb60ec0a96e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:09.545754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c761c-3c79-451c-a4e7-bb60ec0a96e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:09.546208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-37fef6da-ebc3-4874-943c-88b6a575a1f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:09.547679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c761c-3c79-451c-a4e7-bb60ec0a96e9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:09.576192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 867/1731] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:18:09.580357 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c761c-3c79-451c-a4e7-bb60ec0a96e9 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:09.605162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c761c-3c79-451c-a4e7-bb60ec0a96e9 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:09.623428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696ea0f-0845-4fb6-8468-a9ad57e5efdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:09.624115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696ea0f-0845-4fb6-8468-a9ad57e5efdb admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:09.624641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696ea0f-0845-4fb6-8468-a9ad57e5efdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:09.625113 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696ea0f-0845-4fb6-8468-a9ad57e5efdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:09.625996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696ea0f-0845-4fb6-8468-a9ad57e5efdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:09.628161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696ea0f-0845-4fb6-8468-a9ad57e5efdb admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:09.650740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 868/1732] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:09.656991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696ea0f-0845-4fb6-8468-a9ad57e5efdb admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:09.683480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696ea0f-0845-4fb6-8468-a9ad57e5efdb admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:09.700900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48f01c-acf4-4536-ad97-cb581af7fa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:09.701860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48f01c-acf4-4536-ad97-cb581af7fa33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:09.702343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48f01c-acf4-4536-ad97-cb581af7fa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:09.702869 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48f01c-acf4-4536-ad97-cb581af7fa33 admin admin] PATH_INFO: `/v3/projects/480002fe7668489d9dd1a10c7aac6079/users/4f7341730a864dd199e03d76a0c05226/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:09.703714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fef6da-ebc3-4874-943c-88b6a575a1f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:09.705320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fef6da-ebc3-4874-943c-88b6a575a1f6 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:09.738638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 865/1733] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:09 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:18:09.748419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48f01c-acf4-4536-ad97-cb581af7fa33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=480002fe7668489d9dd1a10c7aac6079, user_id=4f7341730a864dd199e03d76a0c05226, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:09.755065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48f01c-acf4-4536-ad97-cb581af7fa33 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:09.795969 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 869/1734] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:09 2022] PUT /identity/v3/projects/480002fe7668489d9dd1a10c7aac6079/users/4f7341730a864dd199e03d76a0c05226/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:18:09.806816 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f4ee2-00a6-435d-8fd4-22187f0653f5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:09.807099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f4ee2-00a6-435d-8fd4-22187f0653f5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:09.807369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f4ee2-00a6-435d-8fd4-22187f0653f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:09.897332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-425f4ee2-00a6-435d-8fd4-22187f0653f5 None None] MFA Rules not processed for user `4f7341730a864dd199e03d76a0c05226`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:09.903638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-425f4ee2-00a6-435d-8fd4-22187f0653f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:10.092027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48f01c-acf4-4536-ad97-cb581af7fa33 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:10.092603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 866/1735] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:10.185964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:10.261748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48f01c-acf4-4536-ad97-cb581af7fa33 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:10.277915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b77e24-0c88-47d1-81d7-3a3870dfcb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:10.278466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b77e24-0c88-47d1-81d7-3a3870dfcb6e admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:10.279822 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b77e24-0c88-47d1-81d7-3a3870dfcb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:10.280257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b77e24-0c88-47d1-81d7-3a3870dfcb6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:10.280873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b77e24-0c88-47d1-81d7-3a3870dfcb6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:10.283072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b77e24-0c88-47d1-81d7-3a3870dfcb6e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:10.346271 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f4ee2-00a6-435d-8fd4-22187f0653f5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:10.346510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 870/1736] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:10 2022] POST /identity/v3/projects => generated 375 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:10.360365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f4ee2-00a6-435d-8fd4-22187f0653f5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:10.451252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:10.515296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fe14b-e664-47f0-b13b-93726e37c394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:10.515940 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fe14b-e664-47f0-b13b-93726e37c394 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:10.516213 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fe14b-e664-47f0-b13b-93726e37c394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:10.516480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29fe14b-e664-47f0-b13b-93726e37c394 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:10.517188 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fe14b-e664-47f0-b13b-93726e37c394 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:10.519256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29fe14b-e664-47f0-b13b-93726e37c394 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:10.590773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 867/1737] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:10 2022] POST /identity/v3/users => generated 294 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:10.594820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b77e24-0c88-47d1-81d7-3a3870dfcb6e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:10.608114 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b77e24-0c88-47d1-81d7-3a3870dfcb6e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:10.619460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d44954-0e25-4cc4-9fca-4085fb28621e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:10.620719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d44954-0e25-4cc4-9fca-4085fb28621e admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:10.621002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d44954-0e25-4cc4-9fca-4085fb28621e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:10.621285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d44954-0e25-4cc4-9fca-4085fb28621e admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:10.624629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d44954-0e25-4cc4-9fca-4085fb28621e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:10.629513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d44954-0e25-4cc4-9fca-4085fb28621e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:10.657561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 871/1738] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:10.659233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fe14b-e664-47f0-b13b-93726e37c394 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:10.677282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29fe14b-e664-47f0-b13b-93726e37c394 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:10.678207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f222f-8b39-43ed-abd4-a1750e997f4b None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:10.678669 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f222f-8b39-43ed-abd4-a1750e997f4b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:10.678949 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f222f-8b39-43ed-abd4-a1750e997f4b None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:10.680580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 872/1739] 10.222.0.75 () {54 vars in 912 bytes} [Fri Sep 23 15:18:10 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:18:10.685773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10aea40-8790-4d1b-8a24-db0850c574bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:10.686493 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10aea40-8790-4d1b-8a24-db0850c574bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:10.686770 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10aea40-8790-4d1b-8a24-db0850c574bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:10.687073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10aea40-8790-4d1b-8a24-db0850c574bf admin admin] PATH_INFO: `/v3/projects/af9de0ddb4ee44a585283564b34d9e82/users/5308c8aad6234d40ad9a5b2c8676f532/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:10.688824 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f75a4f-3d04-4a3d-bcd4-edf4089f5bb6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:10.689141 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f75a4f-3d04-4a3d-bcd4-edf4089f5bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:10.689410 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f75a4f-3d04-4a3d-bcd4-edf4089f5bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:10.733217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10aea40-8790-4d1b-8a24-db0850c574bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9de0ddb4ee44a585283564b34d9e82, user_id=5308c8aad6234d40ad9a5b2c8676f532, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:10.736008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10aea40-8790-4d1b-8a24-db0850c574bf admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:10.781679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 868/1740] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:10 2022] PUT /identity/v3/projects/af9de0ddb4ee44a585283564b34d9e82/users/5308c8aad6234d40ad9a5b2c8676f532/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:10.782988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10aea40-8790-4d1b-8a24-db0850c574bf admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:10.786872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-40f75a4f-3d04-4a3d-bcd4-edf4089f5bb6 None None] MFA Rules not processed for user `02afdedf8d394bbfa977ab7a22fafea7`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:10.791641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-40f75a4f-3d04-4a3d-bcd4-edf4089f5bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:10.951214 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 873/1741] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:18:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:18:10.952529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c96076-2c26-46b6-a198-3efa16d9cbcb None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:10.952838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c96076-2c26-46b6-a198-3efa16d9cbcb None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:10.953107 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c96076-2c26-46b6-a198-3efa16d9cbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:10.958347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10aea40-8790-4d1b-8a24-db0850c574bf admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:10.982740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8870d4-b5e4-46d0-9c9d-16f2097f4692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:10.983377 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8870d4-b5e4-46d0-9c9d-16f2097f4692 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:10.983653 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8870d4-b5e4-46d0-9c9d-16f2097f4692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:10.983921 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8870d4-b5e4-46d0-9c9d-16f2097f4692 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:10.984633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8870d4-b5e4-46d0-9c9d-16f2097f4692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:10.986518 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8870d4-b5e4-46d0-9c9d-16f2097f4692 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:11.066956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-70c96076-2c26-46b6-a198-3efa16d9cbcb None None] MFA Rules not processed for user `e38b853b47b64567b00092eb32201b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:11.070287 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 869/1742] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:11.074792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-70c96076-2c26-46b6-a198-3efa16d9cbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:11.078434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8870d4-b5e4-46d0-9c9d-16f2097f4692 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:11.083912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7e8870d4-b5e4-46d0-9c9d-16f2097f4692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:11.125796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8870d4-b5e4-46d0-9c9d-16f2097f4692 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:11.135966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:11.139427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:11.141435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:11.142358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] PATH_INFO: `/v3/users/23f67753ed624f4a88930d55a48f2d81` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:11.150420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23f67753ed624f4a88930d55a48f2d81)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:11.154597 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:11.277414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 874/1743] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:11.277414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c96076-2c26-46b6-a198-3efa16d9cbcb None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:11.278042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23f67753ed624f4a88930d55a48f2d81'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:11.278042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23f67753ed624f4a88930d55a48f2d81'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:11.290440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-70c96076-2c26-46b6-a198-3efa16d9cbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:11.306574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23f67753ed624f4a88930d55a48f2d81'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:11.320007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c96076-2c26-46b6-a198-3efa16d9cbcb None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:11.329006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20df16-51f9-40e9-a9ee-52b45aef2c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:11.330108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20df16-51f9-40e9-a9ee-52b45aef2c39 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:11.330694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20df16-51f9-40e9-a9ee-52b45aef2c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:11.331819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b20df16-51f9-40e9-a9ee-52b45aef2c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:11.334053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23f67753ed624f4a88930d55a48f2d81'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:11.335181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23f67753ed624f4a88930d55a48f2d81'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:11.335733 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8b20df16-51f9-40e9-a9ee-52b45aef2c39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:11.353181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20df16-51f9-40e9-a9ee-52b45aef2c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:11.355211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 870/1744] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:11 2022] DELETE /identity/v3/users/23f67753ed624f4a88930d55a48f2d81 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:11.356236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:11.358001 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:11.358251 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b20df16-51f9-40e9-a9ee-52b45aef2c39 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:11.376358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:11.390048 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 875/1745] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:11.391308 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20df16-51f9-40e9-a9ee-52b45aef2c39 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:11.397729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8b20df16-51f9-40e9-a9ee-52b45aef2c39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:11.449019 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9510d-26e6-4098-9764-591d7f698b85 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:11.454149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027221c0-44d7-4dba-81a0-97feea7590a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:11.454433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:11.455727 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027221c0-44d7-4dba-81a0-97feea7590a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:11.456126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027221c0-44d7-4dba-81a0-97feea7590a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:11.456554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027221c0-44d7-4dba-81a0-97feea7590a4 admin admin] PATH_INFO: `/v3/projects/af9de0ddb4ee44a585283564b34d9e82/users/5308c8aad6234d40ad9a5b2c8676f532/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:11.459046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027221c0-44d7-4dba-81a0-97feea7590a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9de0ddb4ee44a585283564b34d9e82, user_id=5308c8aad6234d40ad9a5b2c8676f532, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:11.462014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027221c0-44d7-4dba-81a0-97feea7590a4 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:11.475916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 871/1746] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:11 2022] PUT /identity/v3/projects/af9de0ddb4ee44a585283564b34d9e82/users/5308c8aad6234d40ad9a5b2c8676f532/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:11.489720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7003b87-68c2-4936-9142-d4f69a41eb62 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:11.490329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7003b87-68c2-4936-9142-d4f69a41eb62 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:11.490878 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7003b87-68c2-4936-9142-d4f69a41eb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:11.570396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b7003b87-68c2-4936-9142-d4f69a41eb62 None None] MFA Rules not processed for user `5308c8aad6234d40ad9a5b2c8676f532`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:11.574385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b7003b87-68c2-4936-9142-d4f69a41eb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:11.679044 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b20df16-51f9-40e9-a9ee-52b45aef2c39 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:11.794661 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 872/1747] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:18:11.806255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7003b87-68c2-4936-9142-d4f69a41eb62 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:11.809593 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b7003b87-68c2-4936-9142-d4f69a41eb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:11.873067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e72d6d-f4a9-42cc-b5bf-67677219d873 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:11.874057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e72d6d-f4a9-42cc-b5bf-67677219d873 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:11.874474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e72d6d-f4a9-42cc-b5bf-67677219d873 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:11.874880 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e72d6d-f4a9-42cc-b5bf-67677219d873 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:11.878430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-29e72d6d-f4a9-42cc-b5bf-67677219d873 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:11.945675 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7003b87-68c2-4936-9142-d4f69a41eb62 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:11.949987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d241f-6e42-4069-8700-99180281c900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:11.950615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d241f-6e42-4069-8700-99180281c900 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:11.950888 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d241f-6e42-4069-8700-99180281c900 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:11.951152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d241f-6e42-4069-8700-99180281c900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:11.958329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2a6d241f-6e42-4069-8700-99180281c900 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:12.050101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e72d6d-f4a9-42cc-b5bf-67677219d873 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:12.052359 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e72d6d-f4a9-42cc-b5bf-67677219d873 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:12.068124 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:12.096162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 876/1748] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 704 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:12.096560 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d241f-6e42-4069-8700-99180281c900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:12.098139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d241f-6e42-4069-8700-99180281c900 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:12.122023 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 873/1749] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:12.679428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e72d6d-f4a9-42cc-b5bf-67677219d873 service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:12.681266 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-29e72d6d-f4a9-42cc-b5bf-67677219d873 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:12.707265 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33c856-1a68-4e29-ac7c-47ce18110d28 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:12.707554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33c856-1a68-4e29-ac7c-47ce18110d28 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:12.707817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33c856-1a68-4e29-ac7c-47ce18110d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:12.782472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-df33c856-1a68-4e29-ac7c-47ce18110d28 None None] MFA Rules not processed for user `a568b6fd614744abaf5ca41fb26c1f13`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:12.785046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-df33c856-1a68-4e29-ac7c-47ce18110d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:12.926247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:13.020178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 874/1750] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:13.022366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e72d6d-f4a9-42cc-b5bf-67677219d873 service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:13.033913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c6589-f398-415b-999a-f789582073a7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:13.034198 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c6589-f398-415b-999a-f789582073a7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:13.034471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c6589-f398-415b-999a-f789582073a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:13.050822 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:13.051766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:13.055024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:13.055550 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8 admin admin] PATH_INFO: `/v3/projects/a443d1afb1d043e08d6ab0320d4fe92a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:13.058818 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a443d1afb1d043e08d6ab0320d4fe92a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:13.076330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:13.085202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2b1c6589-f398-415b-999a-f789582073a7 None None] MFA Rules not processed for user `a568b6fd614744abaf5ca41fb26c1f13`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:13.087570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2b1c6589-f398-415b-999a-f789582073a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:13.111491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 875/1751] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:13.119784 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c6589-f398-415b-999a-f789582073a7 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:13.123079 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2b1c6589-f398-415b-999a-f789582073a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:13.156388 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c6589-f398-415b-999a-f789582073a7 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:13.172000 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4fbae-a61a-4db7-8d57-8d9ae7a7aaf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:13.180215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4fbae-a61a-4db7-8d57-8d9ae7a7aaf6 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:13.181049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4fbae-a61a-4db7-8d57-8d9ae7a7aaf6 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:13.184692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4fbae-a61a-4db7-8d57-8d9ae7a7aaf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:13.233752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4fbae-a61a-4db7-8d57-8d9ae7a7aaf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:13.239942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4fbae-a61a-4db7-8d57-8d9ae7a7aaf6 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:13.276329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a443d1afb1d043e08d6ab0320d4fe92a', 'request_id': 'req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:13.297712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 876/1752] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:13.300454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4fbae-a61a-4db7-8d57-8d9ae7a7aaf6 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:13.301011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a443d1afb1d043e08d6ab0320d4fe92a', 'request_id': 'req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:13.304752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a0c4fbae-a61a-4db7-8d57-8d9ae7a7aaf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:13.363604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a443d1afb1d043e08d6ab0320d4fe92a', 'request_id': 'req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:13.420129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8 admin admin] The token cache is being invalidate because project a443d1afb1d043e08d6ab0320d4fe92a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:13.422761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:13.425213 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-fb0b8fe0-0300-4f15-93c3-3c12fbfb35f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:13.431362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 877/1753] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:12 2022] DELETE /identity/v3/projects/a443d1afb1d043e08d6ab0320d4fe92a => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:13.435055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea71198-7551-42d1-8cf8-a8d27e3d895b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:13.436168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea71198-7551-42d1-8cf8-a8d27e3d895b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:13.439932 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea71198-7551-42d1-8cf8-a8d27e3d895b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:13.483901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4fbae-a61a-4db7-8d57-8d9ae7a7aaf6 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:13.485959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6ea71198-7551-42d1-8cf8-a8d27e3d895b None None] MFA Rules not processed for user `4cfac62e1c304167abb971815d5da358`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:13.489084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6ea71198-7551-42d1-8cf8-a8d27e3d895b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:13.573103 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:13.585496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22026c-0bd7-4b20-a973-7e09618826c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:13.586610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22026c-0bd7-4b20-a973-7e09618826c5 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:13.586610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22026c-0bd7-4b20-a973-7e09618826c5 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:13.586899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22026c-0bd7-4b20-a973-7e09618826c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:13.589463 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6b22026c-0bd7-4b20-a973-7e09618826c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:13.606907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22026c-0bd7-4b20-a973-7e09618826c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:13.609108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22026c-0bd7-4b20-a973-7e09618826c5 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:13.634231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 877/1754] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:13.635401 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22026c-0bd7-4b20-a973-7e09618826c5 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:13.643566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 878/1755] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:13.643774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6b22026c-0bd7-4b20-a973-7e09618826c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:13.857625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22026c-0bd7-4b20-a973-7e09618826c5 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:13.887052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399f126-440f-4240-b233-c527d0a33515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:13.887928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399f126-440f-4240-b233-c527d0a33515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:13.888347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399f126-440f-4240-b233-c527d0a33515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:13.888856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399f126-440f-4240-b233-c527d0a33515 admin admin] PATH_INFO: `/v3/users/e2fcf9c4bc1443989ae3a6bf793244ad` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:13.891343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399f126-440f-4240-b233-c527d0a33515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2fcf9c4bc1443989ae3a6bf793244ad)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:13.893584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399f126-440f-4240-b233-c527d0a33515 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:13.982059 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6399f126-440f-4240-b233-c527d0a33515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2fcf9c4bc1443989ae3a6bf793244ad'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:13.982938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6399f126-440f-4240-b233-c527d0a33515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2fcf9c4bc1443989ae3a6bf793244ad'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:14.003519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6399f126-440f-4240-b233-c527d0a33515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2fcf9c4bc1443989ae3a6bf793244ad'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:14.023298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6399f126-440f-4240-b233-c527d0a33515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2fcf9c4bc1443989ae3a6bf793244ad'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:14.024118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6399f126-440f-4240-b233-c527d0a33515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2fcf9c4bc1443989ae3a6bf793244ad'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:14.053191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 878/1756] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:13 2022] DELETE /identity/v3/users/e2fcf9c4bc1443989ae3a6bf793244ad => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:14.793371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7a9d2-9caa-4778-91ed-f043ee6f10ba None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:14.793847 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7a9d2-9caa-4778-91ed-f043ee6f10ba None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:14.793925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7a9d2-9caa-4778-91ed-f043ee6f10ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:14.886328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c7a7a9d2-9caa-4778-91ed-f043ee6f10ba None None] MFA Rules not processed for user `4bded8b580f34e08b2890eb3430f4b80`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:14.889866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c7a7a9d2-9caa-4778-91ed-f043ee6f10ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:15.091494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 879/1757] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:15.373791 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3826877-6ace-4bec-a5b2-d60947d0036a None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:15.374725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3826877-6ace-4bec-a5b2-d60947d0036a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:15.375158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3826877-6ace-4bec-a5b2-d60947d0036a None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:15.378666 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 879/1758] 10.222.0.75 () {54 vars in 912 bytes} [Fri Sep 23 15:18:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:18:15.386165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdcf85-07a5-40fd-9fb8-58fd17847545 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:15.386842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdcf85-07a5-40fd-9fb8-58fd17847545 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:15.387301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdcf85-07a5-40fd-9fb8-58fd17847545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:15.469116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cfcdcf85-07a5-40fd-9fb8-58fd17847545 None None] MFA Rules not processed for user `02afdedf8d394bbfa977ab7a22fafea7`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:15.470173 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cfcdcf85-07a5-40fd-9fb8-58fd17847545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:15.504114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3826877-6ace-4bec-a5b2-d60947d0036a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:15.505949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b3826877-6ace-4bec-a5b2-d60947d0036a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:15.514153 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:15.696919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 880/1759] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:18:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:15.697936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdcf85-07a5-40fd-9fb8-58fd17847545 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:15.699525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cfcdcf85-07a5-40fd-9fb8-58fd17847545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:15.868361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3826877-6ace-4bec-a5b2-d60947d0036a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:15.883136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd74206-7261-4bef-9a95-1260a986143b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:15.883805 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd74206-7261-4bef-9a95-1260a986143b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:15.884084 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd74206-7261-4bef-9a95-1260a986143b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:15.884385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd74206-7261-4bef-9a95-1260a986143b admin admin] PATH_INFO: `/v3/projects/6d0d1e294d9f490ba26ea92a9d1ce226` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:15.887628 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd74206-7261-4bef-9a95-1260a986143b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d0d1e294d9f490ba26ea92a9d1ce226)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:15.895197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd74206-7261-4bef-9a95-1260a986143b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:15.916650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdcf85-07a5-40fd-9fb8-58fd17847545 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:15.920420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:15.945015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaba21c-beb8-478f-bb76-03e508876bc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:15.945744 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaba21c-beb8-478f-bb76-03e508876bc6 service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:15.946028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaba21c-beb8-478f-bb76-03e508876bc6 service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:15.946298 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaba21c-beb8-478f-bb76-03e508876bc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:15.993071 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaba21c-beb8-478f-bb76-03e508876bc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:15.994987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaba21c-beb8-478f-bb76-03e508876bc6 service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:16.030731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dcd74206-7261-4bef-9a95-1260a986143b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d0d1e294d9f490ba26ea92a9d1ce226', 'request_id': 'req-dcd74206-7261-4bef-9a95-1260a986143b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:16.046811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 881/1760] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:16.053286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaba21c-beb8-478f-bb76-03e508876bc6 service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:16.057817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ddaba21c-beb8-478f-bb76-03e508876bc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:16.064314 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dcd74206-7261-4bef-9a95-1260a986143b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d0d1e294d9f490ba26ea92a9d1ce226', 'request_id': 'req-dcd74206-7261-4bef-9a95-1260a986143b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:16.093135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dcd74206-7261-4bef-9a95-1260a986143b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d0d1e294d9f490ba26ea92a9d1ce226', 'request_id': 'req-dcd74206-7261-4bef-9a95-1260a986143b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:16.109946 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaba21c-beb8-478f-bb76-03e508876bc6 service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:16.150996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dcd74206-7261-4bef-9a95-1260a986143b admin admin] The token cache is being invalidate because project 6d0d1e294d9f490ba26ea92a9d1ce226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:16.154219 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dcd74206-7261-4bef-9a95-1260a986143b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:16.156638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-dcd74206-7261-4bef-9a95-1260a986143b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:16.176202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 880/1761] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:15 2022] DELETE /identity/v3/projects/6d0d1e294d9f490ba26ea92a9d1ce226 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:18:16.181088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a3255-a51e-4060-814b-597803ebf64d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:16.181370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a3255-a51e-4060-814b-597803ebf64d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:16.181640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a3255-a51e-4060-814b-597803ebf64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:16.259828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-478a3255-a51e-4060-814b-597803ebf64d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:16.261095 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-478a3255-a51e-4060-814b-597803ebf64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:16.354935 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210ab3f-7df6-4ed0-b41b-3a7cc87fbe02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:16.355584 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210ab3f-7df6-4ed0-b41b-3a7cc87fbe02 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:16.355970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210ab3f-7df6-4ed0-b41b-3a7cc87fbe02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:16.356273 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210ab3f-7df6-4ed0-b41b-3a7cc87fbe02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:16.373540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a210ab3f-7df6-4ed0-b41b-3a7cc87fbe02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:16.406899 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210ab3f-7df6-4ed0-b41b-3a7cc87fbe02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:16.408685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210ab3f-7df6-4ed0-b41b-3a7cc87fbe02 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:16.485745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 882/1762] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:16.544395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b78db3-4c62-4f49-8aab-9c9ff58ba3f5 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:16.549202 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b78db3-4c62-4f49-8aab-9c9ff58ba3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:16.549202 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b78db3-4c62-4f49-8aab-9c9ff58ba3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:16.549522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 881/1763] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:16.584060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c3b78db3-4c62-4f49-8aab-9c9ff58ba3f5 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:16.585333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c3b78db3-4c62-4f49-8aab-9c9ff58ba3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:16.611622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a3255-a51e-4060-814b-597803ebf64d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:16.616686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 883/1764] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:16.627218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a3255-a51e-4060-814b-597803ebf64d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:16.638226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442b32f-0f49-47cb-bd74-e31d832780cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:16.639070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442b32f-0f49-47cb-bd74-e31d832780cb admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:16.639507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442b32f-0f49-47cb-bd74-e31d832780cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:16.639926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442b32f-0f49-47cb-bd74-e31d832780cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:16.641189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442b32f-0f49-47cb-bd74-e31d832780cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:18:16.641657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442b32f-0f49-47cb-bd74-e31d832780cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:16.643675 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442b32f-0f49-47cb-bd74-e31d832780cb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:16.679703 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 882/1765] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:18:16.680792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b78db3-4c62-4f49-8aab-9c9ff58ba3f5 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:16.690346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442b32f-0f49-47cb-bd74-e31d832780cb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:16.692297 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3442b32f-0f49-47cb-bd74-e31d832780cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:16.693739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b78db3-4c62-4f49-8aab-9c9ff58ba3f5 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:16.719731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463240bf-0e42-4c02-9a74-d1d622f765e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:16.720480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463240bf-0e42-4c02-9a74-d1d622f765e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:16.720784 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463240bf-0e42-4c02-9a74-d1d622f765e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:16.721048 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463240bf-0e42-4c02-9a74-d1d622f765e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:16.721705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463240bf-0e42-4c02-9a74-d1d622f765e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:16.723597 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463240bf-0e42-4c02-9a74-d1d622f765e1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:16.848974 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 884/1766] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:16 2022] POST /identity/v3/projects => generated 371 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:16.854533 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463240bf-0e42-4c02-9a74-d1d622f765e1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:16.892372 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442b32f-0f49-47cb-bd74-e31d832780cb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:16.912119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:17.052437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463240bf-0e42-4c02-9a74-d1d622f765e1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:17.072266 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddea2d-6375-4948-a183-21f5326c53a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:17.079495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddea2d-6375-4948-a183-21f5326c53a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:17.079824 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddea2d-6375-4948-a183-21f5326c53a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:17.080180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddea2d-6375-4948-a183-21f5326c53a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:17.083994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:17.084165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ddea2d-6375-4948-a183-21f5326c53a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:17.086050 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ddea2d-6375-4948-a183-21f5326c53a4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:17.129516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa24e6-1c1a-4695-a8f6-440f4ce6f48b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:17.130324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa24e6-1c1a-4695-a8f6-440f4ce6f48b service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:17.130624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa24e6-1c1a-4695-a8f6-440f4ce6f48b service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:17.130982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa24e6-1c1a-4695-a8f6-440f4ce6f48b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:17.140747 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-99aa24e6-1c1a-4695-a8f6-440f4ce6f48b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:17.150766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 885/1767] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:16 2022] POST /identity/v3/users => generated 286 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:17.155257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddea2d-6375-4948-a183-21f5326c53a4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:17.177867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddea2d-6375-4948-a183-21f5326c53a4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:17.189566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022aed-c40b-4352-b3df-edf7acd53ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:17.190516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022aed-c40b-4352-b3df-edf7acd53ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:17.191132 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022aed-c40b-4352-b3df-edf7acd53ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:17.191552 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022aed-c40b-4352-b3df-edf7acd53ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:17.192379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a022aed-c40b-4352-b3df-edf7acd53ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:17.194450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a022aed-c40b-4352-b3df-edf7acd53ea4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:17.220048 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 886/1768] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:17.225173 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022aed-c40b-4352-b3df-edf7acd53ea4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:17.256036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022aed-c40b-4352-b3df-edf7acd53ea4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:17.274829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77890298-7feb-4a58-ae7d-69553baf2f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:17.283195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77890298-7feb-4a58-ae7d-69553baf2f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:17.283709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77890298-7feb-4a58-ae7d-69553baf2f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:17.284153 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77890298-7feb-4a58-ae7d-69553baf2f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:17.285221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77890298-7feb-4a58-ae7d-69553baf2f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:17.288260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77890298-7feb-4a58-ae7d-69553baf2f91 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:17.333075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 887/1769] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:17 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:17.334356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77890298-7feb-4a58-ae7d-69553baf2f91 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:17.366354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77890298-7feb-4a58-ae7d-69553baf2f91 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:17.379312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa24e6-1c1a-4695-a8f6-440f4ce6f48b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:17.382250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa24e6-1c1a-4695-a8f6-440f4ce6f48b service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:17.388508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a30e28-0a79-4113-b55f-ad37778f8f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:17.390069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a30e28-0a79-4113-b55f-ad37778f8f17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:17.392294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a30e28-0a79-4113-b55f-ad37778f8f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:17.396135 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a30e28-0a79-4113-b55f-ad37778f8f17 admin admin] PATH_INFO: `/v3/projects/c89c1fb05f6f4662ac6b0cf26673ed13/users/7937f0c95b1241e9a7bb4914f4f7d391/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:17.437561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 883/1770] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 748 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:18:17.448960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a30e28-0a79-4113-b55f-ad37778f8f17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89c1fb05f6f4662ac6b0cf26673ed13, user_id=7937f0c95b1241e9a7bb4914f4f7d391, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:17.457263 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a30e28-0a79-4113-b55f-ad37778f8f17 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:17.490709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 888/1771] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:17 2022] PUT /identity/v3/projects/c89c1fb05f6f4662ac6b0cf26673ed13/users/7937f0c95b1241e9a7bb4914f4f7d391/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:17.501976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ea959-d0b8-4cc3-8b09-f36779779100 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:17.502402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ea959-d0b8-4cc3-8b09-f36779779100 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:17.503095 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ea959-d0b8-4cc3-8b09-f36779779100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:17.560959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3b4ea959-d0b8-4cc3-8b09-f36779779100 None None] MFA Rules not processed for user `7937f0c95b1241e9a7bb4914f4f7d391`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:17.562886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3b4ea959-d0b8-4cc3-8b09-f36779779100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:17.779966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 884/1772] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:17.785155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a30e28-0a79-4113-b55f-ad37778f8f17 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:17.917229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045cc96-4e22-4207-83a3-8a5d2f103f43 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:17.917627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045cc96-4e22-4207-83a3-8a5d2f103f43 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:17.918001 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045cc96-4e22-4207-83a3-8a5d2f103f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:18.028866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a30e28-0a79-4113-b55f-ad37778f8f17 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:18.044957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c44531-715e-49e7-9afa-0242e48dc12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:18.049269 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c44531-715e-49e7-9afa-0242e48dc12f admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:18.055567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c44531-715e-49e7-9afa-0242e48dc12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:18.056732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c44531-715e-49e7-9afa-0242e48dc12f admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:18.057629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c44531-715e-49e7-9afa-0242e48dc12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:18.059772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c44531-715e-49e7-9afa-0242e48dc12f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:18.073957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1045cc96-4e22-4207-83a3-8a5d2f103f43 None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:18.077277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1045cc96-4e22-4207-83a3-8a5d2f103f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:18.175388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 889/1773] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:17 2022] POST /identity/v3/projects => generated 373 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:18.176563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c44531-715e-49e7-9afa-0242e48dc12f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:18.339779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:18.418293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 885/1774] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:18:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:18.483889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:18.512821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c44531-715e-49e7-9afa-0242e48dc12f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:18.518664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea0cf2-deb3-445d-8079-e0b61e4436f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:18.519853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea0cf2-deb3-445d-8079-e0b61e4436f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:18.520081 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea0cf2-deb3-445d-8079-e0b61e4436f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:18.520518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea0cf2-deb3-445d-8079-e0b61e4436f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:18.522297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea0cf2-deb3-445d-8079-e0b61e4436f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:18.524483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea0cf2-deb3-445d-8079-e0b61e4436f2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:18.591097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 890/1775] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:18 2022] POST /identity/v3/users => generated 293 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:18.595639 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045cc96-4e22-4207-83a3-8a5d2f103f43 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:18.630117 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045cc96-4e22-4207-83a3-8a5d2f103f43 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:18.664919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df61514-28d0-4dfb-aa62-d843d4e26132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:18.666318 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df61514-28d0-4dfb-aa62-d843d4e26132 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:18.666593 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df61514-28d0-4dfb-aa62-d843d4e26132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:18.666854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df61514-28d0-4dfb-aa62-d843d4e26132 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:18.667510 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df61514-28d0-4dfb-aa62-d843d4e26132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:18.703424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df61514-28d0-4dfb-aa62-d843d4e26132 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:18.761481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 886/1776] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:18.766226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea0cf2-deb3-445d-8079-e0b61e4436f2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:18.782496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea0cf2-deb3-445d-8079-e0b61e4436f2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:18.813898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e481379-2e88-4d70-87a3-1ed2bd803f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:18.814733 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e481379-2e88-4d70-87a3-1ed2bd803f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:18.815131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e481379-2e88-4d70-87a3-1ed2bd803f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:18.815494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e481379-2e88-4d70-87a3-1ed2bd803f12 admin admin] PATH_INFO: `/v3/projects/a69fbd84f7b24d7581c987a78e471107/users/f7c40a8283844922949f1299cc5b95f9/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:18.898962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e481379-2e88-4d70-87a3-1ed2bd803f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a69fbd84f7b24d7581c987a78e471107, user_id=f7c40a8283844922949f1299cc5b95f9, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:18.906855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e481379-2e88-4d70-87a3-1ed2bd803f12 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:18.950409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df61514-28d0-4dfb-aa62-d843d4e26132 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:18.956451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0df61514-28d0-4dfb-aa62-d843d4e26132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:18.968715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 891/1777] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:18 2022] PUT /identity/v3/projects/a69fbd84f7b24d7581c987a78e471107/users/f7c40a8283844922949f1299cc5b95f9/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:18.970024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e481379-2e88-4d70-87a3-1ed2bd803f12 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:19.260009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e481379-2e88-4d70-87a3-1ed2bd803f12 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:19.264932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df61514-28d0-4dfb-aa62-d843d4e26132 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:19.266806 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3e7d0-0df8-4086-8dce-ab6b5442a760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:19.269163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3e7d0-0df8-4086-8dce-ab6b5442a760 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:19.269760 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3e7d0-0df8-4086-8dce-ab6b5442a760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:19.270592 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3e7d0-0df8-4086-8dce-ab6b5442a760 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:19.271915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3e7d0-0df8-4086-8dce-ab6b5442a760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:19.299843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3e7d0-0df8-4086-8dce-ab6b5442a760 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:19.345949 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 892/1778] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:19.350231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ceb416-d0d0-49b3-870d-ea13e680bddb None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:19.353218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ceb416-d0d0-49b3-870d-ea13e680bddb None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:19.353661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ceb416-d0d0-49b3-870d-ea13e680bddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:19.410221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-21ceb416-d0d0-49b3-870d-ea13e680bddb None None] MFA Rules not processed for user `e38b853b47b64567b00092eb32201b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:19.418462 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-21ceb416-d0d0-49b3-870d-ea13e680bddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:19.483241 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628a495-d41f-4dec-99f7-b60b929a571d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:19.484981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628a495-d41f-4dec-99f7-b60b929a571d service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:19.495463 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628a495-d41f-4dec-99f7-b60b929a571d service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:19.496088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628a495-d41f-4dec-99f7-b60b929a571d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:19.565263 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 893/1779] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:19.566168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ceb416-d0d0-49b3-870d-ea13e680bddb None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:19.590560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ceb416-d0d0-49b3-870d-ea13e680bddb None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:19.592112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628a495-d41f-4dec-99f7-b60b929a571d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:19.594773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628a495-d41f-4dec-99f7-b60b929a571d service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:19.604446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287b241-c284-4375-9992-52be79d45b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:19.605445 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287b241-c284-4375-9992-52be79d45b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:19.605850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287b241-c284-4375-9992-52be79d45b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:19.606250 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287b241-c284-4375-9992-52be79d45b6b admin admin] PATH_INFO: `/v3/projects/a69fbd84f7b24d7581c987a78e471107/users/f7c40a8283844922949f1299cc5b95f9/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:19.609890 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287b241-c284-4375-9992-52be79d45b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a69fbd84f7b24d7581c987a78e471107, user_id=f7c40a8283844922949f1299cc5b95f9, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:19.631559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287b241-c284-4375-9992-52be79d45b6b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:19.634068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 887/1780] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:18:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 15:18:19.635541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628a495-d41f-4dec-99f7-b60b929a571d service swift] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:19.638090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2628a495-d41f-4dec-99f7-b60b929a571d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:19.692242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 894/1781] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:19 2022] PUT /identity/v3/projects/a69fbd84f7b24d7581c987a78e471107/users/f7c40a8283844922949f1299cc5b95f9/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:19.701677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:19.703200 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd6a51-c2ff-4837-9f30-119f0a878128 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:19.703597 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd6a51-c2ff-4837-9f30-119f0a878128 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:19.703980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd6a51-c2ff-4837-9f30-119f0a878128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:19.721017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628a495-d41f-4dec-99f7-b60b929a571d service swift] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:19.809835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-dabd6a51-c2ff-4837-9f30-119f0a878128 None None] MFA Rules not processed for user `f7c40a8283844922949f1299cc5b95f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:19.811645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dabd6a51-c2ff-4837-9f30-119f0a878128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:19.944832 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d5b8e-8d56-4c2a-98cc-b7be052e8ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:19.945732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d5b8e-8d56-4c2a-98cc-b7be052e8ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:19.946199 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d5b8e-8d56-4c2a-98cc-b7be052e8ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:19.946770 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d5b8e-8d56-4c2a-98cc-b7be052e8ca1 admin admin] PATH_INFO: `/v3/users/4f7341730a864dd199e03d76a0c05226` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:19.948562 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d5b8e-8d56-4c2a-98cc-b7be052e8ca1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f7341730a864dd199e03d76a0c05226)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:19.953177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d5b8e-8d56-4c2a-98cc-b7be052e8ca1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:19.982513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:20.067656 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c81d5b8e-8d56-4c2a-98cc-b7be052e8ca1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f7341730a864dd199e03d76a0c05226'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:20.069110 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c81d5b8e-8d56-4c2a-98cc-b7be052e8ca1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f7341730a864dd199e03d76a0c05226'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:20.088361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c81d5b8e-8d56-4c2a-98cc-b7be052e8ca1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f7341730a864dd199e03d76a0c05226'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:20.123976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c81d5b8e-8d56-4c2a-98cc-b7be052e8ca1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f7341730a864dd199e03d76a0c05226'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:20.124981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c81d5b8e-8d56-4c2a-98cc-b7be052e8ca1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f7341730a864dd199e03d76a0c05226'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:20.159966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 895/1782] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:20.177127 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd6a51-c2ff-4837-9f30-119f0a878128 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:20.178761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dabd6a51-c2ff-4837-9f30-119f0a878128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:20.211407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 888/1783] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:19 2022] DELETE /identity/v3/users/4f7341730a864dd199e03d76a0c05226 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:20.216939 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3198a90-a34a-4758-9cc5-ce1ec3d84763 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:20.217237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3198a90-a34a-4758-9cc5-ce1ec3d84763 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:20.217508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3198a90-a34a-4758-9cc5-ce1ec3d84763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:20.250175 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f3198a90-a34a-4758-9cc5-ce1ec3d84763 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:20.256659 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f3198a90-a34a-4758-9cc5-ce1ec3d84763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:20.331638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd6a51-c2ff-4837-9f30-119f0a878128 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:20.470769 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a891e0-307d-42ac-aed7-9e3e45733b08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:20.471665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a891e0-307d-42ac-aed7-9e3e45733b08 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:20.472102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a891e0-307d-42ac-aed7-9e3e45733b08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:20.472513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a891e0-307d-42ac-aed7-9e3e45733b08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:20.475127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 889/1784] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:20.506853 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3198a90-a34a-4758-9cc5-ce1ec3d84763 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:20.509957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f3198a90-a34a-4758-9cc5-ce1ec3d84763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:20.596294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a891e0-307d-42ac-aed7-9e3e45733b08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:20.609910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a891e0-307d-42ac-aed7-9e3e45733b08 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:20.615828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:20.631900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 896/1785] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:18:20.663942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3198a90-a34a-4758-9cc5-ce1ec3d84763 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:20.670509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57dc6f-57ec-4433-9534-044f6ce826a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:20.671338 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57dc6f-57ec-4433-9534-044f6ce826a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:20.671924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57dc6f-57ec-4433-9534-044f6ce826a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:20.673366 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57dc6f-57ec-4433-9534-044f6ce826a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:20.687102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57dc6f-57ec-4433-9534-044f6ce826a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:20.690209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57dc6f-57ec-4433-9534-044f6ce826a3 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:20.718681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 890/1786] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:20.821476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a891e0-307d-42ac-aed7-9e3e45733b08 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:20.838787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a891e0-307d-42ac-aed7-9e3e45733b08 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:20.849154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b79638-3b41-4d50-a6aa-9226333fddd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:20.850011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b79638-3b41-4d50-a6aa-9226333fddd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:20.850442 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b79638-3b41-4d50-a6aa-9226333fddd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:20.850869 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b79638-3b41-4d50-a6aa-9226333fddd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:20.857073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d2b79638-3b41-4d50-a6aa-9226333fddd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:20.951306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:21.064467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b79638-3b41-4d50-a6aa-9226333fddd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:21.066526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b79638-3b41-4d50-a6aa-9226333fddd3 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:21.080438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 897/1787] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:21.710555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57dc6f-57ec-4433-9534-044f6ce826a3 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:21.713040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bb57dc6f-57ec-4433-9534-044f6ce826a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:21.744185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57dc6f-57ec-4433-9534-044f6ce826a3 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:21.779895 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8bb38-b120-432a-960a-973236302450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:21.780555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8bb38-b120-432a-960a-973236302450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:21.781009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8bb38-b120-432a-960a-973236302450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:21.781285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8bb38-b120-432a-960a-973236302450 admin admin] PATH_INFO: `/v3/projects/480002fe7668489d9dd1a10c7aac6079` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:21.782550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8bb38-b120-432a-960a-973236302450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=480002fe7668489d9dd1a10c7aac6079)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:21.787627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8bb38-b120-432a-960a-973236302450 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:21.908012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-66c8bb38-b120-432a-960a-973236302450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480002fe7668489d9dd1a10c7aac6079', 'request_id': 'req-66c8bb38-b120-432a-960a-973236302450'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:21.918757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-66c8bb38-b120-432a-960a-973236302450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480002fe7668489d9dd1a10c7aac6079', 'request_id': 'req-66c8bb38-b120-432a-960a-973236302450'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:21.927563 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-66c8bb38-b120-432a-960a-973236302450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480002fe7668489d9dd1a10c7aac6079', 'request_id': 'req-66c8bb38-b120-432a-960a-973236302450'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:21.939523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-66c8bb38-b120-432a-960a-973236302450 admin admin] The token cache is being invalidate because project 480002fe7668489d9dd1a10c7aac6079 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:21.939806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-66c8bb38-b120-432a-960a-973236302450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:21.940469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-66c8bb38-b120-432a-960a-973236302450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:21.942078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 891/1788] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:21 2022] DELETE /identity/v3/projects/480002fe7668489d9dd1a10c7aac6079 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:21.946133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b79638-3b41-4d50-a6aa-9226333fddd3 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:21.958435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b79638-3b41-4d50-a6aa-9226333fddd3 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:22.030258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:22.031036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:22.031335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:22.031770 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] PATH_INFO: `/v3/users/5308c8aad6234d40ad9a5b2c8676f532` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:22.033312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5308c8aad6234d40ad9a5b2c8676f532)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:22.035273 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:22.106901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5308c8aad6234d40ad9a5b2c8676f532'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:22.119935 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5308c8aad6234d40ad9a5b2c8676f532'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:22.134718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5308c8aad6234d40ad9a5b2c8676f532'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:22.153896 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5308c8aad6234d40ad9a5b2c8676f532'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:22.156325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5308c8aad6234d40ad9a5b2c8676f532'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:22.165942 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 898/1789] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:21 2022] DELETE /identity/v3/users/5308c8aad6234d40ad9a5b2c8676f532 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:18:22.390331 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339bb88-57d2-411b-812d-ec2b8915dcaf None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:22.390711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339bb88-57d2-411b-812d-ec2b8915dcaf None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:22.391038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339bb88-57d2-411b-812d-ec2b8915dcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:22.448474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f339bb88-57d2-411b-812d-ec2b8915dcaf None None] MFA Rules not processed for user `4bded8b580f34e08b2890eb3430f4b80`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:22.454211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f339bb88-57d2-411b-812d-ec2b8915dcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:22.570325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 892/1790] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:22.692376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:22.695466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:22.793648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:22.819709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794ceb9-2ddd-41a7-b0b3-ecc2a47e7331 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:22.826624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:22.827790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:22.828284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:22.828922 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef admin admin] PATH_INFO: `/v3/projects/af9de0ddb4ee44a585283564b34d9e82` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:22.830562 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=af9de0ddb4ee44a585283564b34d9e82)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:22.832888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:22.902125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af9de0ddb4ee44a585283564b34d9e82', 'request_id': 'req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:22.911579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af9de0ddb4ee44a585283564b34d9e82', 'request_id': 'req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:22.924553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af9de0ddb4ee44a585283564b34d9e82', 'request_id': 'req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:22.945587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef admin admin] The token cache is being invalidate because project af9de0ddb4ee44a585283564b34d9e82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:22.946136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:22.947041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8690b26a-c99d-42f9-909b-5bb79ef4f8ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:22.950004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 899/1791] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:22 2022] DELETE /identity/v3/projects/af9de0ddb4ee44a585283564b34d9e82 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:22.954600 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb95846-73a9-49d5-9103-a1d2bd0f3d88 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:22.955074 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb95846-73a9-49d5-9103-a1d2bd0f3d88 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:22.955172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb95846-73a9-49d5-9103-a1d2bd0f3d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:22.979095 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-fcb95846-73a9-49d5-9103-a1d2bd0f3d88 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:22.980431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fcb95846-73a9-49d5-9103-a1d2bd0f3d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:23.055121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:23.076716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 893/1792] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:23.081024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b186c-63de-4002-add8-fa49e7a5862a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:23.081307 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b186c-63de-4002-add8-fa49e7a5862a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:23.081582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b186c-63de-4002-add8-fa49e7a5862a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:23.107036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-251b186c-63de-4002-add8-fa49e7a5862a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:23.109613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-251b186c-63de-4002-add8-fa49e7a5862a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:23.120396 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 900/1793] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:23.125695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb95846-73a9-49d5-9103-a1d2bd0f3d88 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:23.141473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb95846-73a9-49d5-9103-a1d2bd0f3d88 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:23.148213 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c181b38-8b1c-45fc-88dc-7a85c59d620e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:23.149000 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c181b38-8b1c-45fc-88dc-7a85c59d620e admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:23.149431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c181b38-8b1c-45fc-88dc-7a85c59d620e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:23.149842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c181b38-8b1c-45fc-88dc-7a85c59d620e admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:23.153286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c181b38-8b1c-45fc-88dc-7a85c59d620e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:18:23.153779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c181b38-8b1c-45fc-88dc-7a85c59d620e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:23.155904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c181b38-8b1c-45fc-88dc-7a85c59d620e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:23.181406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 894/1794] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:18:23.181530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b186c-63de-4002-add8-fa49e7a5862a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:23.191638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b186c-63de-4002-add8-fa49e7a5862a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:23.201978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cd3b4-97a3-4451-8d4e-c6dab2bdf42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:23.202618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cd3b4-97a3-4451-8d4e-c6dab2bdf42c admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:23.202889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cd3b4-97a3-4451-8d4e-c6dab2bdf42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:23.203152 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cd3b4-97a3-4451-8d4e-c6dab2bdf42c admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:23.203818 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cd3b4-97a3-4451-8d4e-c6dab2bdf42c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:23.206636 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cd3b4-97a3-4451-8d4e-c6dab2bdf42c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:23.246696 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 901/1795] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:23 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:18:23.250711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c181b38-8b1c-45fc-88dc-7a85c59d620e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:23.261365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c181b38-8b1c-45fc-88dc-7a85c59d620e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:23.341285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a64439-acd0-4ee0-831f-f735a968475f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:23.342026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a64439-acd0-4ee0-831f-f735a968475f admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:23.342369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a64439-acd0-4ee0-831f-f735a968475f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:23.342722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a64439-acd0-4ee0-831f-f735a968475f admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:23.343640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a64439-acd0-4ee0-831f-f735a968475f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:23.346365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a64439-acd0-4ee0-831f-f735a968475f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:23.395494 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 895/1796] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:23 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:23.399583 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cd3b4-97a3-4451-8d4e-c6dab2bdf42c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:23.416335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cd3b4-97a3-4451-8d4e-c6dab2bdf42c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:23.430283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70fe87-3568-4bcd-88fe-279735fe77ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:23.431078 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70fe87-3568-4bcd-88fe-279735fe77ca admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:23.431358 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70fe87-3568-4bcd-88fe-279735fe77ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:23.431624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70fe87-3568-4bcd-88fe-279735fe77ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:23.435208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70fe87-3568-4bcd-88fe-279735fe77ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:23.437754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70fe87-3568-4bcd-88fe-279735fe77ca admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:23.484164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 902/1797] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:23.484681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a64439-acd0-4ee0-831f-f735a968475f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:23.502023 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a64439-acd0-4ee0-831f-f735a968475f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:23.513720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b792d4-eb85-49dc-a27d-ccd7764dfba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:23.514431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b792d4-eb85-49dc-a27d-ccd7764dfba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:23.514706 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b792d4-eb85-49dc-a27d-ccd7764dfba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:23.514981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b792d4-eb85-49dc-a27d-ccd7764dfba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:23.515664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b792d4-eb85-49dc-a27d-ccd7764dfba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:23.519007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b792d4-eb85-49dc-a27d-ccd7764dfba6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:23.552579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 896/1798] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:23.570362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70fe87-3568-4bcd-88fe-279735fe77ca admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:23.586060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70fe87-3568-4bcd-88fe-279735fe77ca admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:23.598864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc625922-50c6-4fc8-b93e-a4b2bf6b75e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:23.600480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc625922-50c6-4fc8-b93e-a4b2bf6b75e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:23.600890 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc625922-50c6-4fc8-b93e-a4b2bf6b75e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:23.601166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc625922-50c6-4fc8-b93e-a4b2bf6b75e7 admin admin] PATH_INFO: `/v3/projects/ad8d7292fb544d889f20493adca08cf8/users/b787261c2d9b4ae09f1f449dc8b92057/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:23.621242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc625922-50c6-4fc8-b93e-a4b2bf6b75e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8d7292fb544d889f20493adca08cf8, user_id=b787261c2d9b4ae09f1f449dc8b92057, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:23.624196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc625922-50c6-4fc8-b93e-a4b2bf6b75e7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:23.651743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 903/1799] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:23 2022] PUT /identity/v3/projects/ad8d7292fb544d889f20493adca08cf8/users/b787261c2d9b4ae09f1f449dc8b92057/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:23.653797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20c2e4-1e86-412f-9e1a-79e1a5bbfca0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:23.654317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20c2e4-1e86-412f-9e1a-79e1a5bbfca0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:23.654589 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20c2e4-1e86-412f-9e1a-79e1a5bbfca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:23.695544 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-8c20c2e4-1e86-412f-9e1a-79e1a5bbfca0 None None] MFA Rules not processed for user `b787261c2d9b4ae09f1f449dc8b92057`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:23.696894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8c20c2e4-1e86-412f-9e1a-79e1a5bbfca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:23.807380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 897/1800] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:18:23.815364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc625922-50c6-4fc8-b93e-a4b2bf6b75e7 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:23.818143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-bc625922-50c6-4fc8-b93e-a4b2bf6b75e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:23.950427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc625922-50c6-4fc8-b93e-a4b2bf6b75e7 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:23.956660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1da59-25b3-4640-8822-c194d9764123 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:23.957647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1da59-25b3-4640-8822-c194d9764123 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:23.958170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1da59-25b3-4640-8822-c194d9764123 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:23.958641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1da59-25b3-4640-8822-c194d9764123 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:23.969235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1da59-25b3-4640-8822-c194d9764123 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:23.971404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1da59-25b3-4640-8822-c194d9764123 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:23.997807 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 904/1801] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:24.770730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3b5bf-3248-471e-b182-c66d119faa9a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:24.771624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3b5bf-3248-471e-b182-c66d119faa9a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:24.772153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3b5bf-3248-471e-b182-c66d119faa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:24.839760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e3b3b5bf-3248-471e-b182-c66d119faa9a None None] MFA Rules not processed for user `02afdedf8d394bbfa977ab7a22fafea7`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:24.845923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e3b3b5bf-3248-471e-b182-c66d119faa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:24.974404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:24.989251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 898/1802] 10.222.0.64 () {58 vars in 1015 bytes} [Fri Sep 23 15:18:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:25.063523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1da59-25b3-4640-8822-c194d9764123 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:25.073244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8ab1da59-25b3-4640-8822-c194d9764123 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:25.123973 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1da59-25b3-4640-8822-c194d9764123 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:25.146142 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3e3ca-7a2c-4203-b58f-fc61b523b50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:25.147041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3e3ca-7a2c-4203-b58f-fc61b523b50a service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:25.147465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3e3ca-7a2c-4203-b58f-fc61b523b50a service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:25.153463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3e3ca-7a2c-4203-b58f-fc61b523b50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:25.193318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3e3ca-7a2c-4203-b58f-fc61b523b50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:25.205248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3e3ca-7a2c-4203-b58f-fc61b523b50a service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:25.233836 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 905/1803] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:26.109093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3b5bf-3248-471e-b182-c66d119faa9a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:26.110601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e3b3b5bf-3248-471e-b182-c66d119faa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:26.313533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3b5bf-3248-471e-b182-c66d119faa9a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:26.354289 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:26.355463 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:26.356347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:26.356959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] PATH_INFO: `/v3/users/9d0a6813289b421c876db35565144d8c` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:26.358732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d0a6813289b421c876db35565144d8c)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:26.362438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:26.467029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d0a6813289b421c876db35565144d8c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:26.470232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d0a6813289b421c876db35565144d8c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:26.490290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d0a6813289b421c876db35565144d8c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:26.518487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d0a6813289b421c876db35565144d8c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:26.519598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d0a6813289b421c876db35565144d8c'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:26.578442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 899/1804] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:26 2022] DELETE /identity/v3/users/9d0a6813289b421c876db35565144d8c => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:26.582840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d687b-76e0-495b-95d5-90033a7dd786 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:26.583717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d687b-76e0-495b-95d5-90033a7dd786 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:26.584137 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d687b-76e0-495b-95d5-90033a7dd786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:26.669998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-200d687b-76e0-495b-95d5-90033a7dd786 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:26.674158 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-200d687b-76e0-495b-95d5-90033a7dd786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:26.683762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:26.924666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 906/1805] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:27.033834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:27.035745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:27.298276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35938d3a-51a2-4c02-8270-bf31d9ec00bd admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:27.309915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b396141-bb4e-42f4-a2e8-c31595118754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:27.310553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b396141-bb4e-42f4-a2e8-c31595118754 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:27.312419 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b396141-bb4e-42f4-a2e8-c31595118754 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:27.313197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b396141-bb4e-42f4-a2e8-c31595118754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:27.360409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b396141-bb4e-42f4-a2e8-c31595118754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:27.364771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b396141-bb4e-42f4-a2e8-c31595118754 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:27.399246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 900/1806] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:27.515247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d687b-76e0-495b-95d5-90033a7dd786 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:27.516655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-200d687b-76e0-495b-95d5-90033a7dd786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:27.556721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d687b-76e0-495b-95d5-90033a7dd786 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:27.569610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52f77b-4706-4a56-811f-8de79eb02bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:27.572061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52f77b-4706-4a56-811f-8de79eb02bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:27.572333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52f77b-4706-4a56-811f-8de79eb02bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:27.572645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52f77b-4706-4a56-811f-8de79eb02bba admin admin] PATH_INFO: `/v3/users/b787261c2d9b4ae09f1f449dc8b92057` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:27.582360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52f77b-4706-4a56-811f-8de79eb02bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=b787261c2d9b4ae09f1f449dc8b92057)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:27.586881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52f77b-4706-4a56-811f-8de79eb02bba admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:27.697975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a52f77b-4706-4a56-811f-8de79eb02bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b787261c2d9b4ae09f1f449dc8b92057'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:27.718415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a52f77b-4706-4a56-811f-8de79eb02bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b787261c2d9b4ae09f1f449dc8b92057'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:27.748786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a52f77b-4706-4a56-811f-8de79eb02bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b787261c2d9b4ae09f1f449dc8b92057'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:27.766276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a52f77b-4706-4a56-811f-8de79eb02bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b787261c2d9b4ae09f1f449dc8b92057'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:27.769917 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a52f77b-4706-4a56-811f-8de79eb02bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b787261c2d9b4ae09f1f449dc8b92057'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:27.783292 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 907/1807] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:27 2022] DELETE /identity/v3/users/b787261c2d9b4ae09f1f449dc8b92057 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:18:27.792041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cece82-6565-42fd-b25a-595be01e08c1 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:27.792329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cece82-6565-42fd-b25a-595be01e08c1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:27.792613 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cece82-6565-42fd-b25a-595be01e08c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:27.828090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-01cece82-6565-42fd-b25a-595be01e08c1 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:27.830191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-01cece82-6565-42fd-b25a-595be01e08c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:27.865845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:27.966971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece2d79-a31a-4fa8-b75f-15b6e7c43881 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:27.967254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece2d79-a31a-4fa8-b75f-15b6e7c43881 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:27.967518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece2d79-a31a-4fa8-b75f-15b6e7c43881 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:27.971692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 901/1808] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:27.971826 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 908/1809] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:18:27.990571 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cece82-6565-42fd-b25a-595be01e08c1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:27.995062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-01cece82-6565-42fd-b25a-595be01e08c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:27.995668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901ac16-6370-430c-baa4-dabe9cb3d941 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:27.995979 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901ac16-6370-430c-baa4-dabe9cb3d941 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:27.996249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901ac16-6370-430c-baa4-dabe9cb3d941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:28.033611 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8901ac16-6370-430c-baa4-dabe9cb3d941 None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:28.033611 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8901ac16-6370-430c-baa4-dabe9cb3d941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:28.113689 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cece82-6565-42fd-b25a-595be01e08c1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:28.117745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b80cea-8e62-4709-b175-1cc70c521c60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:28.118473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b80cea-8e62-4709-b175-1cc70c521c60 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:28.118886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b80cea-8e62-4709-b175-1cc70c521c60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:28.119248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b80cea-8e62-4709-b175-1cc70c521c60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:28.130237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b80cea-8e62-4709-b175-1cc70c521c60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:28.132444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b80cea-8e62-4709-b175-1cc70c521c60 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:28.146181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 902/1810] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:28.213551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 909/1811] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:28.242192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b80cea-8e62-4709-b175-1cc70c521c60 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:28.243960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e3b80cea-8e62-4709-b175-1cc70c521c60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:28.262626 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b80cea-8e62-4709-b175-1cc70c521c60 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:28.269902 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b395f-466f-4224-9df1-8bedaff237f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:28.270839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b395f-466f-4224-9df1-8bedaff237f3 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:28.271235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b395f-466f-4224-9df1-8bedaff237f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:28.271718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b395f-466f-4224-9df1-8bedaff237f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:28.282719 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b395f-466f-4224-9df1-8bedaff237f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:28.284948 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b395f-466f-4224-9df1-8bedaff237f3 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:28.299077 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 903/1812] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:28.592435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901ac16-6370-430c-baa4-dabe9cb3d941 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:28.594536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8901ac16-6370-430c-baa4-dabe9cb3d941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:28.613300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901ac16-6370-430c-baa4-dabe9cb3d941 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:28.622256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:28.622914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:28.623365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:28.623772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] PATH_INFO: `/v3/projects/0f71446c610743abb96c197797ff80d1` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:28.625262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f71446c610743abb96c197797ff80d1)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:28.627422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:28.638177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:28.692308 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f71446c610743abb96c197797ff80d1', 'request_id': 'req-f39ec488-b2cc-4d71-9907-b92fcbfad923'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:28.703341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f71446c610743abb96c197797ff80d1', 'request_id': 'req-f39ec488-b2cc-4d71-9907-b92fcbfad923'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:28.712721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f71446c610743abb96c197797ff80d1', 'request_id': 'req-f39ec488-b2cc-4d71-9907-b92fcbfad923'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:28.738247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] The token cache is being invalidate because project 0f71446c610743abb96c197797ff80d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:28.738538 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:28.741927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:28.744172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 910/1813] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:28 2022] DELETE /identity/v3/projects/0f71446c610743abb96c197797ff80d1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:28.748094 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b395f-466f-4224-9df1-8bedaff237f3 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:28.763259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b395f-466f-4224-9df1-8bedaff237f3 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:28.835181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dddd975-059b-4f40-b215-e5138f98407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:28.836161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddd975-059b-4f40-b215-e5138f98407d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:28.836414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddd975-059b-4f40-b215-e5138f98407d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:28.836709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddd975-059b-4f40-b215-e5138f98407d admin admin] PATH_INFO: `/v3/users/15f79a38d3934a9084ca04b3c8d6379a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:28.837854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dddd975-059b-4f40-b215-e5138f98407d admin admin] RBAC: Authorizing `identity:delete_user(user_id=15f79a38d3934a9084ca04b3c8d6379a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:28.840041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dddd975-059b-4f40-b215-e5138f98407d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:28.864970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:28.866638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:28.880952 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ec488-b2cc-4d71-9907-b92fcbfad923 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:28.885763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bba678-0553-4ccc-a4b8-05f361d22380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:28.886472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bba678-0553-4ccc-a4b8-05f361d22380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:28.886745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bba678-0553-4ccc-a4b8-05f361d22380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:28.887009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bba678-0553-4ccc-a4b8-05f361d22380 admin admin] PATH_INFO: `/v3/projects/ad8d7292fb544d889f20493adca08cf8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:28.888565 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bba678-0553-4ccc-a4b8-05f361d22380 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8d7292fb544d889f20493adca08cf8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:28.890486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bba678-0553-4ccc-a4b8-05f361d22380 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:28.935934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3dddd975-059b-4f40-b215-e5138f98407d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15f79a38d3934a9084ca04b3c8d6379a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:28.937523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3dddd975-059b-4f40-b215-e5138f98407d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15f79a38d3934a9084ca04b3c8d6379a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:28.945253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:28.947527 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3dddd975-059b-4f40-b215-e5138f98407d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15f79a38d3934a9084ca04b3c8d6379a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:28.964661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-98bba678-0553-4ccc-a4b8-05f361d22380 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8d7292fb544d889f20493adca08cf8', 'request_id': 'req-98bba678-0553-4ccc-a4b8-05f361d22380'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:28.968985 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3dddd975-059b-4f40-b215-e5138f98407d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15f79a38d3934a9084ca04b3c8d6379a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:28.969823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3dddd975-059b-4f40-b215-e5138f98407d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15f79a38d3934a9084ca04b3c8d6379a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:28.972885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-98bba678-0553-4ccc-a4b8-05f361d22380 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8d7292fb544d889f20493adca08cf8', 'request_id': 'req-98bba678-0553-4ccc-a4b8-05f361d22380'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:28.981093 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-98bba678-0553-4ccc-a4b8-05f361d22380 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8d7292fb544d889f20493adca08cf8', 'request_id': 'req-98bba678-0553-4ccc-a4b8-05f361d22380'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:28.991540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 904/1814] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:28 2022] DELETE /identity/v3/users/15f79a38d3934a9084ca04b3c8d6379a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:29.020725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-98bba678-0553-4ccc-a4b8-05f361d22380 admin admin] The token cache is being invalidate because project ad8d7292fb544d889f20493adca08cf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:29.020725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-98bba678-0553-4ccc-a4b8-05f361d22380 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:29.021126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-98bba678-0553-4ccc-a4b8-05f361d22380 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:29.023315 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 911/1815] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:28 2022] DELETE /identity/v3/projects/ad8d7292fb544d889f20493adca08cf8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:18:29.028615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575576a-d1fb-45a2-8c58-56f130a3c5dc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.029046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575576a-d1fb-45a2-8c58-56f130a3c5dc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.029332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575576a-d1fb-45a2-8c58-56f130a3c5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.054302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5575576a-d1fb-45a2-8c58-56f130a3c5dc None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:29.055525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5575576a-d1fb-45a2-8c58-56f130a3c5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:29.152814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 905/1816] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:29.156269 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fa6df-adf9-4abd-b296-138b256c26e8 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.156572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fa6df-adf9-4abd-b296-138b256c26e8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.157041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fa6df-adf9-4abd-b296-138b256c26e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.177733 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5a0fa6df-adf9-4abd-b296-138b256c26e8 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:29.179117 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5a0fa6df-adf9-4abd-b296-138b256c26e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:29.187771 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 912/1817] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:18:29.192518 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575576a-d1fb-45a2-8c58-56f130a3c5dc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:29.205013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575576a-d1fb-45a2-8c58-56f130a3c5dc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:29.211659 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902244b-aece-42cc-a7a7-3c5a076adcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:29.212508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902244b-aece-42cc-a7a7-3c5a076adcf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.212935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902244b-aece-42cc-a7a7-3c5a076adcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.213819 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902244b-aece-42cc-a7a7-3c5a076adcf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.214946 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902244b-aece-42cc-a7a7-3c5a076adcf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:18:29.215596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902244b-aece-42cc-a7a7-3c5a076adcf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:29.218499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902244b-aece-42cc-a7a7-3c5a076adcf8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:29.229492 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea2314-8749-416c-877d-3fc05b1bf380 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.229774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea2314-8749-416c-877d-3fc05b1bf380 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.230249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea2314-8749-416c-877d-3fc05b1bf380 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.232344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 913/1818] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:18:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:18:29.244658 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 906/1819] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:18:29.245998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea2314-8749-416c-877d-3fc05b1bf380 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:29.256476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a437d-7db8-4a09-af64-e3cfbea52524 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.257498 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a437d-7db8-4a09-af64-e3cfbea52524 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.258169 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a437d-7db8-4a09-af64-e3cfbea52524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.266306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea2314-8749-416c-877d-3fc05b1bf380 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:29.274238 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a691d-80f9-425f-af76-f8c39d16cc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:29.276792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a691d-80f9-425f-af76-f8c39d16cc70 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.277140 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a691d-80f9-425f-af76-f8c39d16cc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.277484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a691d-80f9-425f-af76-f8c39d16cc70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.279420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a691d-80f9-425f-af76-f8c39d16cc70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:29.293871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a691d-80f9-425f-af76-f8c39d16cc70 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:29.321827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-157a437d-7db8-4a09-af64-e3cfbea52524 None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:29.324328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-157a437d-7db8-4a09-af64-e3cfbea52524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:29.359128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 914/1820] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:29 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:18:29.363265 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a691d-80f9-425f-af76-f8c39d16cc70 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:29.417242 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 907/1821] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:29.431186 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a437d-7db8-4a09-af64-e3cfbea52524 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:29.433029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-157a437d-7db8-4a09-af64-e3cfbea52524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:29.448850 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a437d-7db8-4a09-af64-e3cfbea52524 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:29.449792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:29.465305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a691d-80f9-425f-af76-f8c39d16cc70 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:29.472087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fde26-c767-419f-924b-ea404137364c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:29.472819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fde26-c767-419f-924b-ea404137364c admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.473703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fde26-c767-419f-924b-ea404137364c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.473979 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fde26-c767-419f-924b-ea404137364c admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.474647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fde26-c767-419f-924b-ea404137364c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:29.476996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fde26-c767-419f-924b-ea404137364c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:29.502634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 915/1822] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:29 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:18:29.503401 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fde26-c767-419f-924b-ea404137364c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:29.505055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1e0fde26-c767-419f-924b-ea404137364c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:29.506654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:29.517406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fde26-c767-419f-924b-ea404137364c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:29.522849 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:29.523505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.523785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.524052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28 admin admin] PATH_INFO: `/v3/projects/14a33604d47a45b8b25b87be0fc326cb` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.525190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a33604d47a45b8b25b87be0fc326cb)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:29.527115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:29.527930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0e104-2538-4717-a720-34d724e5b332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:29.528545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0e104-2538-4717-a720-34d724e5b332 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.528848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0e104-2538-4717-a720-34d724e5b332 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.529113 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0e104-2538-4717-a720-34d724e5b332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.538751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0e104-2538-4717-a720-34d724e5b332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:29.540361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0e104-2538-4717-a720-34d724e5b332 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:29.551708 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 908/1823] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:29.552661 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0e104-2538-4717-a720-34d724e5b332 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:29.564541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0e104-2538-4717-a720-34d724e5b332 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:29.569603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8243808-b3a4-4d6c-878c-0ac8a177c91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:29.570332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8243808-b3a4-4d6c-878c-0ac8a177c91d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.570643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8243808-b3a4-4d6c-878c-0ac8a177c91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.570919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8243808-b3a4-4d6c-878c-0ac8a177c91d admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.571659 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8243808-b3a4-4d6c-878c-0ac8a177c91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:29.573502 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8243808-b3a4-4d6c-878c-0ac8a177c91d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:29.585100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 909/1824] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:29.589292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8243808-b3a4-4d6c-878c-0ac8a177c91d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:29.595938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a33604d47a45b8b25b87be0fc326cb', 'request_id': 'req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:29.602213 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8243808-b3a4-4d6c-878c-0ac8a177c91d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:29.603300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a33604d47a45b8b25b87be0fc326cb', 'request_id': 'req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:29.608998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a33604d47a45b8b25b87be0fc326cb', 'request_id': 'req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:29.624113 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28 admin admin] The token cache is being invalidate because project 14a33604d47a45b8b25b87be0fc326cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:29.624406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:29.625106 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a3b2ac6-257b-4b82-9260-6e2df1eeff28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:29.627083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 916/1825] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:29 2022] DELETE /identity/v3/projects/14a33604d47a45b8b25b87be0fc326cb => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:29.632044 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b67204-c833-4420-9382-ba022b558350 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.632313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b67204-c833-4420-9382-ba022b558350 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.632579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b67204-c833-4420-9382-ba022b558350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.653139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-32b67204-c833-4420-9382-ba022b558350 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:29.654432 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-32b67204-c833-4420-9382-ba022b558350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:29.663363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765eea52-3c08-4e2e-b87f-2dcf26443efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:29.664102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765eea52-3c08-4e2e-b87f-2dcf26443efd admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.664476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765eea52-3c08-4e2e-b87f-2dcf26443efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.664871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765eea52-3c08-4e2e-b87f-2dcf26443efd admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.665528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765eea52-3c08-4e2e-b87f-2dcf26443efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:29.667163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765eea52-3c08-4e2e-b87f-2dcf26443efd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:29.685229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 910/1826] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:29.689813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765eea52-3c08-4e2e-b87f-2dcf26443efd admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:29.691552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-765eea52-3c08-4e2e-b87f-2dcf26443efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:29.706400 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765eea52-3c08-4e2e-b87f-2dcf26443efd admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:29.711681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7653e-f74b-41ea-a65f-3dccc042c407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:29.712397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7653e-f74b-41ea-a65f-3dccc042c407 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.712711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7653e-f74b-41ea-a65f-3dccc042c407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.712982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7653e-f74b-41ea-a65f-3dccc042c407 admin admin] PATH_INFO: `/v3/projects/17ab30d675e244559a0cb2f7c87a4f5b/users/55d8a79666da49f0998a9e0eab779aa2/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.719391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 917/1827] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:18:29.727427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b06944-a906-4e36-a44d-be35df4f69b3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.727708 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b06944-a906-4e36-a44d-be35df4f69b3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.727976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b06944-a906-4e36-a44d-be35df4f69b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.731322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7653e-f74b-41ea-a65f-3dccc042c407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ab30d675e244559a0cb2f7c87a4f5b, user_id=55d8a79666da49f0998a9e0eab779aa2, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:29.734205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e7653e-f74b-41ea-a65f-3dccc042c407 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:29.749106 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f8b06944-a906-4e36-a44d-be35df4f69b3 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:29.750296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f8b06944-a906-4e36-a44d-be35df4f69b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:29.750843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 911/1828] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:29 2022] PUT /identity/v3/projects/17ab30d675e244559a0cb2f7c87a4f5b/users/55d8a79666da49f0998a9e0eab779aa2/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:18:29.762881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beee60d-e047-4572-8f17-03c631a68f69 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.763272 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beee60d-e047-4572-8f17-03c631a68f69 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.763533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beee60d-e047-4572-8f17-03c631a68f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.794178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1beee60d-e047-4572-8f17-03c631a68f69 None None] MFA Rules not processed for user `55d8a79666da49f0998a9e0eab779aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:29.795371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1beee60d-e047-4572-8f17-03c631a68f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:29.827259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 918/1829] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:18:29.833485 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b06944-a906-4e36-a44d-be35df4f69b3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:29.839106 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:29.842912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b06944-a906-4e36-a44d-be35df4f69b3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:29.849189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0d87a-3a5f-4dc5-ac16-467600100a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:29.850381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0d87a-3a5f-4dc5-ac16-467600100a4f admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.850853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0d87a-3a5f-4dc5-ac16-467600100a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.851170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0d87a-3a5f-4dc5-ac16-467600100a4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.852041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0d87a-3a5f-4dc5-ac16-467600100a4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:18:29.852726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0d87a-3a5f-4dc5-ac16-467600100a4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:29.854786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0d87a-3a5f-4dc5-ac16-467600100a4f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:29.863581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 912/1830] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:29.867647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 919/1831] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:18:29.871991 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beee60d-e047-4572-8f17-03c631a68f69 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:29.875293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0d87a-3a5f-4dc5-ac16-467600100a4f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:29.877032 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6ab0d87a-3a5f-4dc5-ac16-467600100a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:29.884350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beee60d-e047-4572-8f17-03c631a68f69 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:29.888881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917d8c-6ece-4799-825b-66cb2905809a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:29.889482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917d8c-6ece-4799-825b-66cb2905809a admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:29.889750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917d8c-6ece-4799-825b-66cb2905809a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:29.890011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917d8c-6ece-4799-825b-66cb2905809a admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:29.890662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b917d8c-6ece-4799-825b-66cb2905809a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:29.892778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b917d8c-6ece-4799-825b-66cb2905809a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:29.923605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 913/1832] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:29 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:18:29.927841 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917d8c-6ece-4799-825b-66cb2905809a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:29.934208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:29.963936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0d87a-3a5f-4dc5-ac16-467600100a4f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:29.997929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917d8c-6ece-4799-825b-66cb2905809a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:30.003715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a758bb-b0d1-47c8-93b2-f7f4b0630f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:30.004344 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a758bb-b0d1-47c8-93b2-f7f4b0630f22 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:30.004678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a758bb-b0d1-47c8-93b2-f7f4b0630f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:30.004962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a758bb-b0d1-47c8-93b2-f7f4b0630f22 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:30.005985 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a758bb-b0d1-47c8-93b2-f7f4b0630f22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:30.008063 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a758bb-b0d1-47c8-93b2-f7f4b0630f22 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:30.018229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595d05c-aef6-4930-a2eb-873208cc506f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac783fbe95ea4b74bd46af415c9734fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:30.019127 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595d05c-aef6-4930-a2eb-873208cc506f service glance] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:30.019477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595d05c-aef6-4930-a2eb-873208cc506f service glance] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:30.020201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595d05c-aef6-4930-a2eb-873208cc506f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:30.036638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 914/1833] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:29 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:30.037831 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a758bb-b0d1-47c8-93b2-f7f4b0630f22 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:30.047713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a758bb-b0d1-47c8-93b2-f7f4b0630f22 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:30.052804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806603a-1b10-4c11-b701-f49f5a18b7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:30.053403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806603a-1b10-4c11-b701-f49f5a18b7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:30.053713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806603a-1b10-4c11-b701-f49f5a18b7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:30.053979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806603a-1b10-4c11-b701-f49f5a18b7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:30.054608 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806603a-1b10-4c11-b701-f49f5a18b7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:30.056248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806603a-1b10-4c11-b701-f49f5a18b7f6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:30.070011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 915/1834] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:18:30.071332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806603a-1b10-4c11-b701-f49f5a18b7f6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:30.082067 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806603a-1b10-4c11-b701-f49f5a18b7f6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:30.087052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595d05c-aef6-4930-a2eb-873208cc506f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:30.087205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5558eca-b128-47bf-8f7c-019601f547f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:30.087735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5558eca-b128-47bf-8f7c-019601f547f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:30.088015 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5558eca-b128-47bf-8f7c-019601f547f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:30.088281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5558eca-b128-47bf-8f7c-019601f547f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:30.088533 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595d05c-aef6-4930-a2eb-873208cc506f service glance] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:30.088979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5558eca-b128-47bf-8f7c-019601f547f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:30.090818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5558eca-b128-47bf-8f7c-019601f547f8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:30.097943 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 920/1835] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:18:30.102155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 916/1836] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:18:30.106665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595d05c-aef6-4930-a2eb-873208cc506f service glance] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:30.117878 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595d05c-aef6-4930-a2eb-873208cc506f service glance] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:30.122898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c9b01-5446-40cc-b353-145f1dea9fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:30.123563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c9b01-5446-40cc-b353-145f1dea9fef admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:30.123837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c9b01-5446-40cc-b353-145f1dea9fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:30.124241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c9b01-5446-40cc-b353-145f1dea9fef admin admin] PATH_INFO: `/v3/projects/28c7a94160ca4c9e88272a4c09b042d1/users/4b456fa66bd748fe82d6caada10f3cc6/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:30.140554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c9b01-5446-40cc-b353-145f1dea9fef admin admin] RBAC: Authorizing `identity:create_grant(project_id=28c7a94160ca4c9e88272a4c09b042d1, user_id=4b456fa66bd748fe82d6caada10f3cc6, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:30.143310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c9b01-5446-40cc-b353-145f1dea9fef admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:30.172336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 921/1837] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:30 2022] PUT /identity/v3/projects/28c7a94160ca4c9e88272a4c09b042d1/users/4b456fa66bd748fe82d6caada10f3cc6/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:18:30.183099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaefdc-8ff5-4774-8a21-52797cd92529 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:30.183494 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaefdc-8ff5-4774-8a21-52797cd92529 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:30.183838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaefdc-8ff5-4774-8a21-52797cd92529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:30.223085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-bacaefdc-8ff5-4774-8a21-52797cd92529 None None] MFA Rules not processed for user `4b456fa66bd748fe82d6caada10f3cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:30.224305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bacaefdc-8ff5-4774-8a21-52797cd92529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:30.300620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 917/1838] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:30.305424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c9b01-5446-40cc-b353-145f1dea9fef admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:30.327586 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:30.380302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c9b01-5446-40cc-b353-145f1dea9fef admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:30.386756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e9f9a-06d5-43a4-b06d-c82a1d30cf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:30.388123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e9f9a-06d5-43a4-b06d-c82a1d30cf05 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:30.388276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e9f9a-06d5-43a4-b06d-c82a1d30cf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:30.388510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e9f9a-06d5-43a4-b06d-c82a1d30cf05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:30.391102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e9f9a-06d5-43a4-b06d-c82a1d30cf05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:30.393073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e9f9a-06d5-43a4-b06d-c82a1d30cf05 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:30.427170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 922/1839] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:30 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:30.431499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaefdc-8ff5-4774-8a21-52797cd92529 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:30.443589 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaefdc-8ff5-4774-8a21-52797cd92529 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:30.509115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0bac7-ac91-4d04-94ef-147cc903dd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:30.509740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0bac7-ac91-4d04-94ef-147cc903dd6c admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:30.510015 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0bac7-ac91-4d04-94ef-147cc903dd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:30.510284 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0bac7-ac91-4d04-94ef-147cc903dd6c admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:30.510942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0bac7-ac91-4d04-94ef-147cc903dd6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:30.512943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0bac7-ac91-4d04-94ef-147cc903dd6c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:30.525060 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:30.539474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 918/1840] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:30 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:30.543860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e9f9a-06d5-43a4-b06d-c82a1d30cf05 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:30.556978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e9f9a-06d5-43a4-b06d-c82a1d30cf05 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:30.563159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50477ed9-364b-4446-84e9-0466208d54f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:30.563818 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50477ed9-364b-4446-84e9-0466208d54f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:30.564150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50477ed9-364b-4446-84e9-0466208d54f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:30.564628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50477ed9-364b-4446-84e9-0466208d54f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:30.565533 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50477ed9-364b-4446-84e9-0466208d54f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:30.567420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50477ed9-364b-4446-84e9-0466208d54f9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:30.580231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 923/1841] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:18:30.584654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0bac7-ac91-4d04-94ef-147cc903dd6c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:30.596285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0bac7-ac91-4d04-94ef-147cc903dd6c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:30.602425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9835a-f239-4b76-9625-e51caf6ffa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:30.603741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9835a-f239-4b76-9625-e51caf6ffa85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:30.604027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9835a-f239-4b76-9625-e51caf6ffa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:30.604305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9835a-f239-4b76-9625-e51caf6ffa85 admin admin] PATH_INFO: `/v3/projects/583b53f958ba4240856aeea7501ac63b/users/615de3aa71064f8ea2f48fb0d4f6e42d/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:30.642551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9835a-f239-4b76-9625-e51caf6ffa85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=583b53f958ba4240856aeea7501ac63b, user_id=615de3aa71064f8ea2f48fb0d4f6e42d, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:30.657809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9835a-f239-4b76-9625-e51caf6ffa85 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:30.688834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 919/1842] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:30 2022] PUT /identity/v3/projects/583b53f958ba4240856aeea7501ac63b/users/615de3aa71064f8ea2f48fb0d4f6e42d/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:30.690333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50477ed9-364b-4446-84e9-0466208d54f9 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:30.704044 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50477ed9-364b-4446-84e9-0466208d54f9 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:30.843817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2961d4-0731-4daa-9a2e-afdda65e6f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:30.848810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2961d4-0731-4daa-9a2e-afdda65e6f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:30.849282 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2961d4-0731-4daa-9a2e-afdda65e6f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:30.849823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2961d4-0731-4daa-9a2e-afdda65e6f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:30.851055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2961d4-0731-4daa-9a2e-afdda65e6f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:30.862082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2961d4-0731-4daa-9a2e-afdda65e6f0d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:30.907239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 924/1843] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:30.911313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9835a-f239-4b76-9625-e51caf6ffa85 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:30.930267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9835a-f239-4b76-9625-e51caf6ffa85 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:30.941329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517df29-dd2d-4b2e-833a-9d13ec9accda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:30.942035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517df29-dd2d-4b2e-833a-9d13ec9accda admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:30.942328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517df29-dd2d-4b2e-833a-9d13ec9accda admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:30.942553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517df29-dd2d-4b2e-833a-9d13ec9accda admin admin] PATH_INFO: `/v3/projects/583b53f958ba4240856aeea7501ac63b/users/615de3aa71064f8ea2f48fb0d4f6e42d/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:30.944657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517df29-dd2d-4b2e-833a-9d13ec9accda admin admin] RBAC: Authorizing `identity:create_grant(project_id=583b53f958ba4240856aeea7501ac63b, user_id=615de3aa71064f8ea2f48fb0d4f6e42d, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:30.947387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517df29-dd2d-4b2e-833a-9d13ec9accda admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:30.962695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 920/1844] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:30 2022] PUT /identity/v3/projects/583b53f958ba4240856aeea7501ac63b/users/615de3aa71064f8ea2f48fb0d4f6e42d/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:30.974812 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e957431-06aa-4ae9-bb6a-c2f09aa9656b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:30.975371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e957431-06aa-4ae9-bb6a-c2f09aa9656b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:30.975738 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e957431-06aa-4ae9-bb6a-c2f09aa9656b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:31.009649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4e957431-06aa-4ae9-bb6a-c2f09aa9656b None None] MFA Rules not processed for user `615de3aa71064f8ea2f48fb0d4f6e42d`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:31.010927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4e957431-06aa-4ae9-bb6a-c2f09aa9656b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:31.088878 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 925/1845] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:31.100285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517df29-dd2d-4b2e-833a-9d13ec9accda admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:31.102190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8517df29-dd2d-4b2e-833a-9d13ec9accda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:31.167476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517df29-dd2d-4b2e-833a-9d13ec9accda admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:31.171363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181792be-789f-4917-a11e-59fe0497c0c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:31.171986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181792be-789f-4917-a11e-59fe0497c0c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:31.172259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181792be-789f-4917-a11e-59fe0497c0c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:31.172521 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181792be-789f-4917-a11e-59fe0497c0c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:31.183799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181792be-789f-4917-a11e-59fe0497c0c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:31.185286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181792be-789f-4917-a11e-59fe0497c0c3 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:31.199384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 921/1846] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:31.884404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e957431-06aa-4ae9-bb6a-c2f09aa9656b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:31.900221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:31.950281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e957431-06aa-4ae9-bb6a-c2f09aa9656b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:31.955367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:31.956201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:31.956698 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:31.957002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] PATH_INFO: `/v3/users/4b456fa66bd748fe82d6caada10f3cc6` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:31.958502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b456fa66bd748fe82d6caada10f3cc6)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:31.960528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:32.013476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b456fa66bd748fe82d6caada10f3cc6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:32.021012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b456fa66bd748fe82d6caada10f3cc6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:32.033866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b456fa66bd748fe82d6caada10f3cc6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:32.042500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b456fa66bd748fe82d6caada10f3cc6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:32.043106 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b456fa66bd748fe82d6caada10f3cc6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:32.049456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 926/1847] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:31 2022] DELETE /identity/v3/users/4b456fa66bd748fe82d6caada10f3cc6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:32.054026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b103c0-fe03-4f10-be25-6e54d585a449 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:32.054674 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b103c0-fe03-4f10-be25-6e54d585a449 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:32.054942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b103c0-fe03-4f10-be25-6e54d585a449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:32.076915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-81b103c0-fe03-4f10-be25-6e54d585a449 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:32.078124 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-81b103c0-fe03-4f10-be25-6e54d585a449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:32.147465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 922/1848] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:32.157776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:32.159481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:32.220258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195aeadc-7ab7-4bc3-ab23-9a91bd97a831 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:32.223884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ae88d-ae1f-454a-81f8-bd6b3e0dc1e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:32.224519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ae88d-ae1f-454a-81f8-bd6b3e0dc1e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:32.224825 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ae88d-ae1f-454a-81f8-bd6b3e0dc1e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:32.225096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ae88d-ae1f-454a-81f8-bd6b3e0dc1e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:32.234306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ae88d-ae1f-454a-81f8-bd6b3e0dc1e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:32.236189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ae88d-ae1f-454a-81f8-bd6b3e0dc1e4 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:32.246535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 927/1849] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:32.615526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b103c0-fe03-4f10-be25-6e54d585a449 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:32.619097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-81b103c0-fe03-4f10-be25-6e54d585a449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:32.666974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b103c0-fe03-4f10-be25-6e54d585a449 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:32.683293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:32.683925 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:32.684196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:32.684462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] PATH_INFO: `/v3/projects/28c7a94160ca4c9e88272a4c09b042d1` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:32.686114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28c7a94160ca4c9e88272a4c09b042d1)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:32.689813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:32.729303 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21b30f-4418-46d0-bc08-2a852d17857b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:32.729586 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21b30f-4418-46d0-bc08-2a852d17857b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:32.729862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21b30f-4418-46d0-bc08-2a852d17857b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:32.752628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-fb21b30f-4418-46d0-bc08-2a852d17857b None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:32.754007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fb21b30f-4418-46d0-bc08-2a852d17857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:32.773503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28c7a94160ca4c9e88272a4c09b042d1', 'request_id': 'req-ba136bd3-9909-4d82-b511-11c1bc422411'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:32.776148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:32.784889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28c7a94160ca4c9e88272a4c09b042d1', 'request_id': 'req-ba136bd3-9909-4d82-b511-11c1bc422411'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:32.789983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:32.797200 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28c7a94160ca4c9e88272a4c09b042d1', 'request_id': 'req-ba136bd3-9909-4d82-b511-11c1bc422411'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:32.823067 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] The token cache is being invalidate because project 28c7a94160ca4c9e88272a4c09b042d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:32.823359 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:32.824085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:32.828692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 923/1850] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:32 2022] DELETE /identity/v3/projects/28c7a94160ca4c9e88272a4c09b042d1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:32.830509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:32.837893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:32.849052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 928/1851] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:18:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:18:32.878157 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21b30f-4418-46d0-bc08-2a852d17857b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:32.882353 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fb21b30f-4418-46d0-bc08-2a852d17857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:32.943735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba136bd3-9909-4d82-b511-11c1bc422411 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:32.949125 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:32.949932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:32.950279 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:32.950621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] PATH_INFO: `/v3/users/615de3aa71064f8ea2f48fb0d4f6e42d` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:32.951945 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=615de3aa71064f8ea2f48fb0d4f6e42d)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:32.954076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:33.055399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21b30f-4418-46d0-bc08-2a852d17857b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:33.082964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '615de3aa71064f8ea2f48fb0d4f6e42d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:33.084085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '615de3aa71064f8ea2f48fb0d4f6e42d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:33.112980 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '615de3aa71064f8ea2f48fb0d4f6e42d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:33.133972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '615de3aa71064f8ea2f48fb0d4f6e42d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:33.135747 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '615de3aa71064f8ea2f48fb0d4f6e42d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:33.150492 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d363f-43fb-4710-aba4-98a70eca3dd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:33.151362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d363f-43fb-4710-aba4-98a70eca3dd5 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:33.151901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d363f-43fb-4710-aba4-98a70eca3dd5 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:33.152430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d363f-43fb-4710-aba4-98a70eca3dd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:33.154924 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c50d363f-43fb-4710-aba4-98a70eca3dd5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:33.156683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 924/1852] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:32 2022] DELETE /identity/v3/users/615de3aa71064f8ea2f48fb0d4f6e42d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:33.229249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d363f-43fb-4710-aba4-98a70eca3dd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:33.230941 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d363f-43fb-4710-aba4-98a70eca3dd5 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:33.243013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 929/1853] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:18:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:18:33.820376 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:33.823384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:33.939245 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c484379-a00e-4623-bb48-e97cdc81bac8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:33.945327 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:33.945986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:33.946264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:33.946538 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044 admin admin] PATH_INFO: `/v3/projects/583b53f958ba4240856aeea7501ac63b` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:33.947696 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=583b53f958ba4240856aeea7501ac63b)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:33.949688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:33.951646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:34.027604 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '583b53f958ba4240856aeea7501ac63b', 'request_id': 'req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:34.036468 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '583b53f958ba4240856aeea7501ac63b', 'request_id': 'req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:34.044008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '583b53f958ba4240856aeea7501ac63b', 'request_id': 'req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:34.065711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044 admin admin] The token cache is being invalidate because project 583b53f958ba4240856aeea7501ac63b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:34.066134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:34.066821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c9b2b1e6-0e04-45d2-85d9-c4aca6031044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:34.068764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 925/1854] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:33 2022] DELETE /identity/v3/projects/583b53f958ba4240856aeea7501ac63b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:34.074405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ef27d-ec04-4bc0-9a2a-961c79b1dfed None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:34.075056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ef27d-ec04-4bc0-9a2a-961c79b1dfed None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:34.075056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ef27d-ec04-4bc0-9a2a-961c79b1dfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:34.097564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5f8ef27d-ec04-4bc0-9a2a-961c79b1dfed None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:34.098884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5f8ef27d-ec04-4bc0-9a2a-961c79b1dfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:34.121672 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:34.205687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 930/1855] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:34.214736 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b144b-9963-40f3-886e-c4ed271e0b2f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:34.215275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b144b-9963-40f3-886e-c4ed271e0b2f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:34.215352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b144b-9963-40f3-886e-c4ed271e0b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:34.249940 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-db1b144b-9963-40f3-886e-c4ed271e0b2f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:34.251433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-db1b144b-9963-40f3-886e-c4ed271e0b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:34.282176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ef27d-ec04-4bc0-9a2a-961c79b1dfed None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:34.283188 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 926/1856] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:34.294679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ef27d-ec04-4bc0-9a2a-961c79b1dfed None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:34.302942 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15193a67-26f1-4ad9-9637-206891e9e627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:34.304206 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15193a67-26f1-4ad9-9637-206891e9e627 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:34.304523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15193a67-26f1-4ad9-9637-206891e9e627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:34.304977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15193a67-26f1-4ad9-9637-206891e9e627 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:34.307465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15193a67-26f1-4ad9-9637-206891e9e627 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:18:34.307770 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15193a67-26f1-4ad9-9637-206891e9e627 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:34.309842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15193a67-26f1-4ad9-9637-206891e9e627 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:34.350320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 931/1857] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:18:34.355126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b144b-9963-40f3-886e-c4ed271e0b2f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:34.366759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b144b-9963-40f3-886e-c4ed271e0b2f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:34.373202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bae42-86ce-42fa-888b-f2e5902a117a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:34.374009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bae42-86ce-42fa-888b-f2e5902a117a admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:34.374349 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bae42-86ce-42fa-888b-f2e5902a117a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:34.374916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bae42-86ce-42fa-888b-f2e5902a117a admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:34.375725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bae42-86ce-42fa-888b-f2e5902a117a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:34.377976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bae42-86ce-42fa-888b-f2e5902a117a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:34.422356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 927/1858] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:34 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:34.427191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15193a67-26f1-4ad9-9637-206891e9e627 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:34.438626 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15193a67-26f1-4ad9-9637-206891e9e627 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:34.503073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179ef79-2901-4101-a478-7371b29984c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:34.503692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179ef79-2901-4101-a478-7371b29984c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:34.503964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179ef79-2901-4101-a478-7371b29984c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:34.504293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179ef79-2901-4101-a478-7371b29984c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:34.505086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179ef79-2901-4101-a478-7371b29984c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:34.507101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179ef79-2901-4101-a478-7371b29984c5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:34.533807 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 932/1859] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:34 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:34.538343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bae42-86ce-42fa-888b-f2e5902a117a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:34.549029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bae42-86ce-42fa-888b-f2e5902a117a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:34.565531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8796c-0098-4dbf-b633-b708d17310e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:34.566185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8796c-0098-4dbf-b633-b708d17310e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:34.566458 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8796c-0098-4dbf-b633-b708d17310e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:34.566723 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8796c-0098-4dbf-b633-b708d17310e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:34.567412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8796c-0098-4dbf-b633-b708d17310e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:34.569098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8796c-0098-4dbf-b633-b708d17310e5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:34.580304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 928/1860] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:34.584423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179ef79-2901-4101-a478-7371b29984c5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:34.594983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179ef79-2901-4101-a478-7371b29984c5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:34.599916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3a584-132c-41ae-b284-b9fcaea15f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:34.600525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3a584-132c-41ae-b284-b9fcaea15f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:34.600832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3a584-132c-41ae-b284-b9fcaea15f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:34.601158 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3a584-132c-41ae-b284-b9fcaea15f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:34.601826 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3a584-132c-41ae-b284-b9fcaea15f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:34.603415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3a584-132c-41ae-b284-b9fcaea15f16 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:34.615347 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 933/1861] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:18:34.619444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8796c-0098-4dbf-b633-b708d17310e5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:34.630323 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8796c-0098-4dbf-b633-b708d17310e5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:34.636616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d0444-020c-42f7-8a8a-628b4441925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:34.637446 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d0444-020c-42f7-8a8a-628b4441925c admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:34.637824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d0444-020c-42f7-8a8a-628b4441925c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:34.638162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d0444-020c-42f7-8a8a-628b4441925c admin admin] PATH_INFO: `/v3/projects/0a1b0845fc2e44289528c6fe7a9715e7/users/fa00a47332f64bac93f7450690525e82/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:34.655086 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d0444-020c-42f7-8a8a-628b4441925c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a1b0845fc2e44289528c6fe7a9715e7, user_id=fa00a47332f64bac93f7450690525e82, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:34.657924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d0444-020c-42f7-8a8a-628b4441925c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:34.673025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 929/1862] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:34 2022] PUT /identity/v3/projects/0a1b0845fc2e44289528c6fe7a9715e7/users/fa00a47332f64bac93f7450690525e82/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:34.683374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad35677-da43-4620-9405-7cd5571361d2 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:34.683654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad35677-da43-4620-9405-7cd5571361d2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:34.683924 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad35677-da43-4620-9405-7cd5571361d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:34.712830 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0ad35677-da43-4620-9405-7cd5571361d2 None None] MFA Rules not processed for user `fa00a47332f64bac93f7450690525e82`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:34.714139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0ad35677-da43-4620-9405-7cd5571361d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:34.781535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 934/1863] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:34.786763 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d0444-020c-42f7-8a8a-628b4441925c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:34.860788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d0444-020c-42f7-8a8a-628b4441925c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:34.864478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7fd3c-b5e2-4422-83db-90e10b61e717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:34.865148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7fd3c-b5e2-4422-83db-90e10b61e717 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:34.865418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7fd3c-b5e2-4422-83db-90e10b61e717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:34.865688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7fd3c-b5e2-4422-83db-90e10b61e717 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:34.866325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7fd3c-b5e2-4422-83db-90e10b61e717 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:34.868214 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7fd3c-b5e2-4422-83db-90e10b61e717 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:34.896947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 930/1864] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:34 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:34.901065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad35677-da43-4620-9405-7cd5571361d2 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:34.913263 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad35677-da43-4620-9405-7cd5571361d2 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:34.935343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:34.982073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9c82b-4689-417b-9954-5bb408c1c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:34.982694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9c82b-4689-417b-9954-5bb408c1c6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:34.982967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9c82b-4689-417b-9954-5bb408c1c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:34.983237 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db9c82b-4689-417b-9954-5bb408c1c6d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:34.983874 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9c82b-4689-417b-9954-5bb408c1c6d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:34.985817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db9c82b-4689-417b-9954-5bb408c1c6d1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:35.008887 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 935/1865] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:34 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:35.019650 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7fd3c-b5e2-4422-83db-90e10b61e717 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:35.033614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7fd3c-b5e2-4422-83db-90e10b61e717 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:35.039394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77afae45-838a-4d0d-bfb9-b3ce5cecd003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:35.039934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afae45-838a-4d0d-bfb9-b3ce5cecd003 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:35.040194 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afae45-838a-4d0d-bfb9-b3ce5cecd003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:35.040461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77afae45-838a-4d0d-bfb9-b3ce5cecd003 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:35.041155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77afae45-838a-4d0d-bfb9-b3ce5cecd003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:35.042862 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77afae45-838a-4d0d-bfb9-b3ce5cecd003 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:35.049281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:35.056872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 931/1866] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:35.060929 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9c82b-4689-417b-9954-5bb408c1c6d1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:35.071621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db9c82b-4689-417b-9954-5bb408c1c6d1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:35.076832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328c354-ffb5-4129-8b58-b6301d4e9d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:35.077527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328c354-ffb5-4129-8b58-b6301d4e9d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:35.077760 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328c354-ffb5-4129-8b58-b6301d4e9d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:35.078036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328c354-ffb5-4129-8b58-b6301d4e9d05 admin admin] PATH_INFO: `/v3/projects/0e996a41e6b6404e9fa47898f82db4d5/users/82e59546d4024dccbe2ca76896b0063c/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:35.093645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328c354-ffb5-4129-8b58-b6301d4e9d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e996a41e6b6404e9fa47898f82db4d5, user_id=82e59546d4024dccbe2ca76896b0063c, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:35.096412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328c354-ffb5-4129-8b58-b6301d4e9d05 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:35.113033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 936/1867] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:35 2022] PUT /identity/v3/projects/0e996a41e6b6404e9fa47898f82db4d5/users/82e59546d4024dccbe2ca76896b0063c/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:35.117173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77afae45-838a-4d0d-bfb9-b3ce5cecd003 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:35.127774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77afae45-838a-4d0d-bfb9-b3ce5cecd003 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:35.187408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6110a-c63d-4d89-a303-c56b10baa319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:35.188020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6110a-c63d-4d89-a303-c56b10baa319 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:35.188288 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6110a-c63d-4d89-a303-c56b10baa319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:35.188553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6110a-c63d-4d89-a303-c56b10baa319 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:35.189227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6110a-c63d-4d89-a303-c56b10baa319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:35.191203 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6110a-c63d-4d89-a303-c56b10baa319 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:35.201187 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 932/1868] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:35.205835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328c354-ffb5-4129-8b58-b6301d4e9d05 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:35.217679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328c354-ffb5-4129-8b58-b6301d4e9d05 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:35.222847 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041496db-df6b-448a-8023-9cda3d4d520f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:35.223507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041496db-df6b-448a-8023-9cda3d4d520f admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:35.223779 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041496db-df6b-448a-8023-9cda3d4d520f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:35.224071 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041496db-df6b-448a-8023-9cda3d4d520f admin admin] PATH_INFO: `/v3/projects/0e996a41e6b6404e9fa47898f82db4d5/users/82e59546d4024dccbe2ca76896b0063c/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:35.226050 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041496db-df6b-448a-8023-9cda3d4d520f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e996a41e6b6404e9fa47898f82db4d5, user_id=82e59546d4024dccbe2ca76896b0063c, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:35.228690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041496db-df6b-448a-8023-9cda3d4d520f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:35.241786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 937/1869] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:35 2022] PUT /identity/v3/projects/0e996a41e6b6404e9fa47898f82db4d5/users/82e59546d4024dccbe2ca76896b0063c/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:35.253227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44745f9b-5f2e-4fb4-b813-5dadb25ccc26 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:35.253794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44745f9b-5f2e-4fb4-b813-5dadb25ccc26 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:35.253794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44745f9b-5f2e-4fb4-b813-5dadb25ccc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:35.285498 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-44745f9b-5f2e-4fb4-b813-5dadb25ccc26 None None] MFA Rules not processed for user `82e59546d4024dccbe2ca76896b0063c`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:35.286752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-44745f9b-5f2e-4fb4-b813-5dadb25ccc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:35.365385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 933/1870] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:35.377973 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041496db-df6b-448a-8023-9cda3d4d520f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:35.379795 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-041496db-df6b-448a-8023-9cda3d4d520f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:35.468315 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041496db-df6b-448a-8023-9cda3d4d520f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:35.472935 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8e9ab-531b-472e-a932-9b97f5b16f2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:35.473469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8e9ab-531b-472e-a932-9b97f5b16f2d service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:35.473756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8e9ab-531b-472e-a932-9b97f5b16f2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:35.474009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8e9ab-531b-472e-a932-9b97f5b16f2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:35.492283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8e9ab-531b-472e-a932-9b97f5b16f2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:35.493944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8e9ab-531b-472e-a932-9b97f5b16f2d service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:35.514002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 938/1871] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:35.623378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44745f9b-5f2e-4fb4-b813-5dadb25ccc26 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:35.632757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44745f9b-5f2e-4fb4-b813-5dadb25ccc26 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:35.636409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0915d-3519-4724-9df1-2a483a3814cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:35.637054 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0915d-3519-4724-9df1-2a483a3814cb service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:35.637325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0915d-3519-4724-9df1-2a483a3814cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:35.637588 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0915d-3519-4724-9df1-2a483a3814cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:35.711105 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0915d-3519-4724-9df1-2a483a3814cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:35.712826 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0915d-3519-4724-9df1-2a483a3814cb service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:35.723879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 934/1872] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:36.114929 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8e9ab-531b-472e-a932-9b97f5b16f2d service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:36.196702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8e9ab-531b-472e-a932-9b97f5b16f2d service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:36.201442 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:36.202089 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:36.202409 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:36.202626 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] PATH_INFO: `/v3/users/fa00a47332f64bac93f7450690525e82` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:36.203768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa00a47332f64bac93f7450690525e82)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:36.205822 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:36.232757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:36.278629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa00a47332f64bac93f7450690525e82'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:36.286137 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa00a47332f64bac93f7450690525e82'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:36.298180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa00a47332f64bac93f7450690525e82'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:36.307659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa00a47332f64bac93f7450690525e82'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:36.308491 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa00a47332f64bac93f7450690525e82'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:36.319364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 939/1873] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:36 2022] DELETE /identity/v3/users/fa00a47332f64bac93f7450690525e82 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:36.323984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843f169-2df4-407f-a211-466e57ebaa51 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:36.325209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843f169-2df4-407f-a211-466e57ebaa51 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:36.325482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843f169-2df4-407f-a211-466e57ebaa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:36.351265 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a843f169-2df4-407f-a211-466e57ebaa51 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:36.352668 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a843f169-2df4-407f-a211-466e57ebaa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:36.379011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:36.462151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 935/1874] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:36.472620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:36.474252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:36.555537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70e306-7f6c-42e5-a642-f85f5c47676f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:36.562423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9cafb-c4e7-4c97-9fc4-fd9f688daaf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:36.563159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9cafb-c4e7-4c97-9fc4-fd9f688daaf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:36.563427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9cafb-c4e7-4c97-9fc4-fd9f688daaf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:36.563694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9cafb-c4e7-4c97-9fc4-fd9f688daaf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:36.577737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9cafb-c4e7-4c97-9fc4-fd9f688daaf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:36.579488 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9cafb-c4e7-4c97-9fc4-fd9f688daaf3 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:36.592669 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 940/1875] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:18:36.951075 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843f169-2df4-407f-a211-466e57ebaa51 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:36.952560 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a843f169-2df4-407f-a211-466e57ebaa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:36.966364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843f169-2df4-407f-a211-466e57ebaa51 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:36.971339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:36.972064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:36.973014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:36.973014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] PATH_INFO: `/v3/projects/0a1b0845fc2e44289528c6fe7a9715e7` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:36.973912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a1b0845fc2e44289528c6fe7a9715e7)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:36.975941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:37.043405 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a1b0845fc2e44289528c6fe7a9715e7', 'request_id': 'req-3120011f-7354-4658-b83f-a99fddafe27d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.055620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a1b0845fc2e44289528c6fe7a9715e7', 'request_id': 'req-3120011f-7354-4658-b83f-a99fddafe27d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.064796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a1b0845fc2e44289528c6fe7a9715e7', 'request_id': 'req-3120011f-7354-4658-b83f-a99fddafe27d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.078306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] The token cache is being invalidate because project 0a1b0845fc2e44289528c6fe7a9715e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:37.078906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.079832 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.082749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 936/1876] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:36 2022] DELETE /identity/v3/projects/0a1b0845fc2e44289528c6fe7a9715e7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:37.086981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9cafb-c4e7-4c97-9fc4-fd9f688daaf3 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:37.097937 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9cafb-c4e7-4c97-9fc4-fd9f688daaf3 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:37.192819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7075ef-a098-4996-b5db-d0b971e6f959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:37.193591 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7075ef-a098-4996-b5db-d0b971e6f959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:37.193783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7075ef-a098-4996-b5db-d0b971e6f959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:37.194005 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7075ef-a098-4996-b5db-d0b971e6f959 admin admin] PATH_INFO: `/v3/users/82e59546d4024dccbe2ca76896b0063c` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:37.195182 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7075ef-a098-4996-b5db-d0b971e6f959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82e59546d4024dccbe2ca76896b0063c)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:37.197489 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7075ef-a098-4996-b5db-d0b971e6f959 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:37.255987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:37.262981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bd7075ef-a098-4996-b5db-d0b971e6f959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82e59546d4024dccbe2ca76896b0063c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.270604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bd7075ef-a098-4996-b5db-d0b971e6f959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82e59546d4024dccbe2ca76896b0063c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.282104 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bd7075ef-a098-4996-b5db-d0b971e6f959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82e59546d4024dccbe2ca76896b0063c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.290891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bd7075ef-a098-4996-b5db-d0b971e6f959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82e59546d4024dccbe2ca76896b0063c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.291520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bd7075ef-a098-4996-b5db-d0b971e6f959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82e59546d4024dccbe2ca76896b0063c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.297630 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 941/1877] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:37 2022] DELETE /identity/v3/users/82e59546d4024dccbe2ca76896b0063c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:37.644359 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:37.646706 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:37.670498 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120011f-7354-4658-b83f-a99fddafe27d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:37.798383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be1366-8113-443a-a1e1-0e04725e99ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:37.799443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be1366-8113-443a-a1e1-0e04725e99ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:37.800526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be1366-8113-443a-a1e1-0e04725e99ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:37.801117 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be1366-8113-443a-a1e1-0e04725e99ff admin admin] PATH_INFO: `/v3/projects/0e996a41e6b6404e9fa47898f82db4d5` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:37.805822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be1366-8113-443a-a1e1-0e04725e99ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e996a41e6b6404e9fa47898f82db4d5)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:37.808076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be1366-8113-443a-a1e1-0e04725e99ff admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:37.885555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:37.888959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f1be1366-8113-443a-a1e1-0e04725e99ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e996a41e6b6404e9fa47898f82db4d5', 'request_id': 'req-f1be1366-8113-443a-a1e1-0e04725e99ff'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.908999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f1be1366-8113-443a-a1e1-0e04725e99ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e996a41e6b6404e9fa47898f82db4d5', 'request_id': 'req-f1be1366-8113-443a-a1e1-0e04725e99ff'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.919545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f1be1366-8113-443a-a1e1-0e04725e99ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e996a41e6b6404e9fa47898f82db4d5', 'request_id': 'req-f1be1366-8113-443a-a1e1-0e04725e99ff'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.931569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f1be1366-8113-443a-a1e1-0e04725e99ff admin admin] The token cache is being invalidate because project 0e996a41e6b6404e9fa47898f82db4d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:18:37.931842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f1be1366-8113-443a-a1e1-0e04725e99ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.932678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f1be1366-8113-443a-a1e1-0e04725e99ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:37.934388 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 937/1878] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:18:37 2022] DELETE /identity/v3/projects/0e996a41e6b6404e9fa47898f82db4d5 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:37.939401 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8dcb1-9c49-4526-9b0c-47ac22c7bdc0 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:37.939675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8dcb1-9c49-4526-9b0c-47ac22c7bdc0 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:37.940023 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8dcb1-9c49-4526-9b0c-47ac22c7bdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:37.960516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2dc8dcb1-9c49-4526-9b0c-47ac22c7bdc0 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:37.961680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2dc8dcb1-9c49-4526-9b0c-47ac22c7bdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:38.023305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 942/1879] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:38.031824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab090ad6-7688-4cb2-bcfe-7ee2c9780d10 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.032234 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab090ad6-7688-4cb2-bcfe-7ee2c9780d10 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.032621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab090ad6-7688-4cb2-bcfe-7ee2c9780d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.085369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ab090ad6-7688-4cb2-bcfe-7ee2c9780d10 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:38.086906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ab090ad6-7688-4cb2-bcfe-7ee2c9780d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:38.096804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 938/1880] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:38.101222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8dcb1-9c49-4526-9b0c-47ac22c7bdc0 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:38.110826 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8dcb1-9c49-4526-9b0c-47ac22c7bdc0 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:38.116076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cacda-2d81-4242-b2db-731d34f04424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:38.116730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cacda-2d81-4242-b2db-731d34f04424 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.117116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cacda-2d81-4242-b2db-731d34f04424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.117291 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cacda-2d81-4242-b2db-731d34f04424 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.118086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cacda-2d81-4242-b2db-731d34f04424 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:18:38.118393 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cacda-2d81-4242-b2db-731d34f04424 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:38.119993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761cacda-2d81-4242-b2db-731d34f04424 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:38.132995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 943/1881] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:18:38.137274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab090ad6-7688-4cb2-bcfe-7ee2c9780d10 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:38.146684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab090ad6-7688-4cb2-bcfe-7ee2c9780d10 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:38.151421 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7da69e-614e-4393-884f-6c361dfac079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:38.152112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7da69e-614e-4393-884f-6c361dfac079 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.152494 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7da69e-614e-4393-884f-6c361dfac079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.152811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7da69e-614e-4393-884f-6c361dfac079 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.153600 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7da69e-614e-4393-884f-6c361dfac079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:38.157302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7da69e-614e-4393-884f-6c361dfac079 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:38.184896 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 939/1882] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:38 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:38.189057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cacda-2d81-4242-b2db-731d34f04424 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:38.198964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cacda-2d81-4242-b2db-731d34f04424 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:38.259292 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92943b-ca73-4243-a4c7-29aaee743dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:38.259994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92943b-ca73-4243-a4c7-29aaee743dd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.260337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92943b-ca73-4243-a4c7-29aaee743dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.260658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92943b-ca73-4243-a4c7-29aaee743dd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.261229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92943b-ca73-4243-a4c7-29aaee743dd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:38.263221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92943b-ca73-4243-a4c7-29aaee743dd5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:38.294626 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 944/1883] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:38 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:18:38.295975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7da69e-614e-4393-884f-6c361dfac079 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:38.316760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7da69e-614e-4393-884f-6c361dfac079 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:38.321941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e9317-63c7-41c1-a9fe-e1d9f559ac6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:38.322676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e9317-63c7-41c1-a9fe-e1d9f559ac6f admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.323040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e9317-63c7-41c1-a9fe-e1d9f559ac6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.323375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e9317-63c7-41c1-a9fe-e1d9f559ac6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.324103 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e9317-63c7-41c1-a9fe-e1d9f559ac6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:38.325753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e9317-63c7-41c1-a9fe-e1d9f559ac6f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:38.336963 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 940/1884] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:38.342479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92943b-ca73-4243-a4c7-29aaee743dd5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:38.353782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92943b-ca73-4243-a4c7-29aaee743dd5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:38.358586 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d999424-67f9-4977-a034-2ea1cb26333a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:38.359194 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d999424-67f9-4977-a034-2ea1cb26333a admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.359465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d999424-67f9-4977-a034-2ea1cb26333a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.359731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d999424-67f9-4977-a034-2ea1cb26333a admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.360380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d999424-67f9-4977-a034-2ea1cb26333a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:38.362289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d999424-67f9-4977-a034-2ea1cb26333a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:38.374000 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 945/1885] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:38.378257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e9317-63c7-41c1-a9fe-e1d9f559ac6f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:38.388170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e9317-63c7-41c1-a9fe-e1d9f559ac6f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:38.392813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c3ac0-504c-4518-8761-a94e592dd202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:38.393542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c3ac0-504c-4518-8761-a94e592dd202 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.393802 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c3ac0-504c-4518-8761-a94e592dd202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.394062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c3ac0-504c-4518-8761-a94e592dd202 admin admin] PATH_INFO: `/v3/projects/745d83dfccc54a0e8abb65c6e71e6923/users/bfd7f365f4a4486fa7ca991f5aaafc1a/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.408941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c3ac0-504c-4518-8761-a94e592dd202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=745d83dfccc54a0e8abb65c6e71e6923, user_id=bfd7f365f4a4486fa7ca991f5aaafc1a, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:38.411667 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c3ac0-504c-4518-8761-a94e592dd202 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:38.426913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 941/1886] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:38 2022] PUT /identity/v3/projects/745d83dfccc54a0e8abb65c6e71e6923/users/bfd7f365f4a4486fa7ca991f5aaafc1a/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:18:38.441150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9aaa9-d553-4e3b-ae9b-5c282fb32615 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.441431 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9aaa9-d553-4e3b-ae9b-5c282fb32615 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.441724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9aaa9-d553-4e3b-ae9b-5c282fb32615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.475190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0ec9aaa9-d553-4e3b-ae9b-5c282fb32615 None None] MFA Rules not processed for user `bfd7f365f4a4486fa7ca991f5aaafc1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:38.476413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0ec9aaa9-d553-4e3b-ae9b-5c282fb32615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:38.486030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:38.546215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 946/1887] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:38.551111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c3ac0-504c-4518-8761-a94e592dd202 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:38.620315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c3ac0-504c-4518-8761-a94e592dd202 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:38.625247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ff1b5-423e-4d3d-bef9-dd33fca75912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:38.625851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ff1b5-423e-4d3d-bef9-dd33fca75912 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.626125 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ff1b5-423e-4d3d-bef9-dd33fca75912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.626392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ff1b5-423e-4d3d-bef9-dd33fca75912 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.627039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ff1b5-423e-4d3d-bef9-dd33fca75912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:38.629190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ff1b5-423e-4d3d-bef9-dd33fca75912 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:38.657450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 942/1888] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:18:38 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:38.662559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9aaa9-d553-4e3b-ae9b-5c282fb32615 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:38.674789 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9aaa9-d553-4e3b-ae9b-5c282fb32615 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:38.734202 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e6c62-787d-41b3-bc2b-104cefffd729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:38.734814 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e6c62-787d-41b3-bc2b-104cefffd729 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.735087 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e6c62-787d-41b3-bc2b-104cefffd729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.735352 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e6c62-787d-41b3-bc2b-104cefffd729 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.735985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e6c62-787d-41b3-bc2b-104cefffd729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:38.738292 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e6c62-787d-41b3-bc2b-104cefffd729 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:38.766099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 947/1889] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:18:38 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:18:38.770234 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ff1b5-423e-4d3d-bef9-dd33fca75912 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:38.783140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ff1b5-423e-4d3d-bef9-dd33fca75912 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:38.788812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97625748-c82f-4330-b7df-70c6895da134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:38.789424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97625748-c82f-4330-b7df-70c6895da134 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.789692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97625748-c82f-4330-b7df-70c6895da134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.789958 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97625748-c82f-4330-b7df-70c6895da134 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.790605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97625748-c82f-4330-b7df-70c6895da134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:38.792237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97625748-c82f-4330-b7df-70c6895da134 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:38.803853 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 943/1890] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:18:38.808230 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e6c62-787d-41b3-bc2b-104cefffd729 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:38.819049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e6c62-787d-41b3-bc2b-104cefffd729 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:38.824014 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f9e83-c8a1-4a88-b972-0f5669deed94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:38.824705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f9e83-c8a1-4a88-b972-0f5669deed94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.824982 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f9e83-c8a1-4a88-b972-0f5669deed94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.825246 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f9e83-c8a1-4a88-b972-0f5669deed94 admin admin] PATH_INFO: `/v3/projects/624ab6a265f24ea08f75645d45230b9b/users/4694c1d16ba547edb447d8f2d763a42f/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.840823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f9e83-c8a1-4a88-b972-0f5669deed94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624ab6a265f24ea08f75645d45230b9b, user_id=4694c1d16ba547edb447d8f2d763a42f, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:38.843510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f9e83-c8a1-4a88-b972-0f5669deed94 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:38.858673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 948/1891] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:38 2022] PUT /identity/v3/projects/624ab6a265f24ea08f75645d45230b9b/users/4694c1d16ba547edb447d8f2d763a42f/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:38.864023 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97625748-c82f-4330-b7df-70c6895da134 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:38.873717 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97625748-c82f-4330-b7df-70c6895da134 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:38.916504 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:38.933773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780c4e0-5cd6-4389-8bd8-40c2972f510e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:38.934270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780c4e0-5cd6-4389-8bd8-40c2972f510e admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.934530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780c4e0-5cd6-4389-8bd8-40c2972f510e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.934792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780c4e0-5cd6-4389-8bd8-40c2972f510e admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.935433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780c4e0-5cd6-4389-8bd8-40c2972f510e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:38.937086 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780c4e0-5cd6-4389-8bd8-40c2972f510e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:38.947439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 944/1892] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:18:38.953726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f9e83-c8a1-4a88-b972-0f5669deed94 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:38.964642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f9e83-c8a1-4a88-b972-0f5669deed94 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:38.969820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f633-6d3e-43f1-89e6-07c8f50a4831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:38.970472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f633-6d3e-43f1-89e6-07c8f50a4831 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:38.970746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f633-6d3e-43f1-89e6-07c8f50a4831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:38.971053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f633-6d3e-43f1-89e6-07c8f50a4831 admin admin] PATH_INFO: `/v3/projects/624ab6a265f24ea08f75645d45230b9b/users/4694c1d16ba547edb447d8f2d763a42f/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:38.973115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26f633-6d3e-43f1-89e6-07c8f50a4831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624ab6a265f24ea08f75645d45230b9b, user_id=4694c1d16ba547edb447d8f2d763a42f, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:38.975783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26f633-6d3e-43f1-89e6-07c8f50a4831 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:38.986059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 949/1893] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:18:38 2022] PUT /identity/v3/projects/624ab6a265f24ea08f75645d45230b9b/users/4694c1d16ba547edb447d8f2d763a42f/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:18:39.001419 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab8a8b-6081-45b8-a110-83006e63686b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:39.001752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab8a8b-6081-45b8-a110-83006e63686b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:39.002113 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab8a8b-6081-45b8-a110-83006e63686b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:39.034982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-eeab8a8b-6081-45b8-a110-83006e63686b None None] MFA Rules not processed for user `4694c1d16ba547edb447d8f2d763a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:39.036185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-eeab8a8b-6081-45b8-a110-83006e63686b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:39.112146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 945/1894] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:39.125308 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f633-6d3e-43f1-89e6-07c8f50a4831 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:39.127120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1b26f633-6d3e-43f1-89e6-07c8f50a4831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:39.195976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f633-6d3e-43f1-89e6-07c8f50a4831 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:39.200183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49991415-80d2-46a6-abbe-5799edfd73b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:39.200852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49991415-80d2-46a6-abbe-5799edfd73b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:39.201112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49991415-80d2-46a6-abbe-5799edfd73b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:39.201506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49991415-80d2-46a6-abbe-5799edfd73b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:39.220581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:39.262168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49991415-80d2-46a6-abbe-5799edfd73b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:39.263744 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49991415-80d2-46a6-abbe-5799edfd73b6 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:39.272867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 950/1895] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:49.108501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=43 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:49.268861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=19 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:49.290493 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab8a8b-6081-45b8-a110-83006e63686b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:49.300558 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab8a8b-6081-45b8-a110-83006e63686b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:49.305849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec17c4-76b1-4e03-ad39-38d91258adfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:49.306678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec17c4-76b1-4e03-ad39-38d91258adfb service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:49.306812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec17c4-76b1-4e03-ad39-38d91258adfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:49.306970 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec17c4-76b1-4e03-ad39-38d91258adfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:49.317766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec17c4-76b1-4e03-ad39-38d91258adfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:49.319209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec17c4-76b1-4e03-ad39-38d91258adfb service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:49.338021 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 946/1896] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:57.141450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dc0ef-f288-48c4-b2de-d339ac187d7a None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:57.141450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dc0ef-f288-48c4-b2de-d339ac187d7a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:57.141450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dc0ef-f288-48c4-b2de-d339ac187d7a None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:57.142339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 951/1897] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:18:57.171662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e6ee8-349a-4d1f-ab4f-4446667cf336 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:57.172130 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e6ee8-349a-4d1f-ab4f-4446667cf336 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:57.172451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e6ee8-349a-4d1f-ab4f-4446667cf336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:57.217947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b13e6ee8-349a-4d1f-ab4f-4446667cf336 None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:18:57.220825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b13e6ee8-349a-4d1f-ab4f-4446667cf336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:57.357181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 947/1898] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:18:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:18:57.374708 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36dc0ef-f288-48c4-b2de-d339ac187d7a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:57.376874 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e36dc0ef-f288-48c4-b2de-d339ac187d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:57.416027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36dc0ef-f288-48c4-b2de-d339ac187d7a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:57.424509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c47c2-74e5-417f-97e6-85d9e98e80eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:57.425317 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c47c2-74e5-417f-97e6-85d9e98e80eb service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:57.425902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c47c2-74e5-417f-97e6-85d9e98e80eb service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:57.426466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c47c2-74e5-417f-97e6-85d9e98e80eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:57.443397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c47c2-74e5-417f-97e6-85d9e98e80eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:57.446628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c47c2-74e5-417f-97e6-85d9e98e80eb service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:57.463893 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 952/1899] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:18:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:18:59.766201 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e6ee8-349a-4d1f-ab4f-4446667cf336 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:59.767882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b13e6ee8-349a-4d1f-ab4f-4446667cf336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:18:59.846258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c47c2-74e5-417f-97e6-85d9e98e80eb service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:18:59.854184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e6ee8-349a-4d1f-ab4f-4446667cf336 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:59.868613 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76c959-3909-4357-88d9-7cbacdda8ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:59.868846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c47c2-74e5-417f-97e6-85d9e98e80eb service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:18:59.868929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76c959-3909-4357-88d9-7cbacdda8ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:59.868929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76c959-3909-4357-88d9-7cbacdda8ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:59.868929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76c959-3909-4357-88d9-7cbacdda8ab0 admin admin] PATH_INFO: `/v3/users/19aff7eae5934fd6952c14f556a45104` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:59.868929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76c959-3909-4357-88d9-7cbacdda8ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19aff7eae5934fd6952c14f556a45104)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:59.869209 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2469aea-6b38-4261-af7b-213bb00715c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:18:59.869288 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76c959-3909-4357-88d9-7cbacdda8ab0 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:59.869356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2469aea-6b38-4261-af7b-213bb00715c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:59.869356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2469aea-6b38-4261-af7b-213bb00715c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:59.869356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2469aea-6b38-4261-af7b-213bb00715c9 admin admin] PATH_INFO: `/v3/users/bfd7f365f4a4486fa7ca991f5aaafc1a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:59.869356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2469aea-6b38-4261-af7b-213bb00715c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd7f365f4a4486fa7ca991f5aaafc1a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:18:59.869663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2469aea-6b38-4261-af7b-213bb00715c9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:18:59.921917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5e76c959-3909-4357-88d9-7cbacdda8ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19aff7eae5934fd6952c14f556a45104'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:59.922648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5e76c959-3909-4357-88d9-7cbacdda8ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19aff7eae5934fd6952c14f556a45104'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:59.925761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2469aea-6b38-4261-af7b-213bb00715c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd7f365f4a4486fa7ca991f5aaafc1a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:59.930461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5e76c959-3909-4357-88d9-7cbacdda8ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19aff7eae5934fd6952c14f556a45104'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:59.935211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2469aea-6b38-4261-af7b-213bb00715c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd7f365f4a4486fa7ca991f5aaafc1a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:59.943758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5e76c959-3909-4357-88d9-7cbacdda8ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19aff7eae5934fd6952c14f556a45104'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:59.944479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5e76c959-3909-4357-88d9-7cbacdda8ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19aff7eae5934fd6952c14f556a45104'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:59.951615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2469aea-6b38-4261-af7b-213bb00715c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd7f365f4a4486fa7ca991f5aaafc1a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:59.960561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:18:59.968471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 948/1900] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:59 2022] DELETE /identity/v3/users/19aff7eae5934fd6952c14f556a45104 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:18:59.972798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2469aea-6b38-4261-af7b-213bb00715c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd7f365f4a4486fa7ca991f5aaafc1a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:59.973424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2469aea-6b38-4261-af7b-213bb00715c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd7f365f4a4486fa7ca991f5aaafc1a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:18:59.973884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b389129-7c55-4177-8e96-d9eae11dfb4d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:59.974168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b389129-7c55-4177-8e96-d9eae11dfb4d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:59.974461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b389129-7c55-4177-8e96-d9eae11dfb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:18:59.983579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 953/1901] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:18:59 2022] DELETE /identity/v3/users/bfd7f365f4a4486fa7ca991f5aaafc1a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:18:59.984976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2a09e-fb83-4b30-910e-a40e7a490dee None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:18:59.985261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2a09e-fb83-4b30-910e-a40e7a490dee None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:18:59.985533 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2a09e-fb83-4b30-910e-a40e7a490dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:00.000486 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6b389129-7c55-4177-8e96-d9eae11dfb4d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:00.001784 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6b389129-7c55-4177-8e96-d9eae11dfb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:00.010860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5ba2a09e-fb83-4b30-910e-a40e7a490dee None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:00.012059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5ba2a09e-fb83-4b30-910e-a40e7a490dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:00.067433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 949/1902] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:19:00.084286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 954/1903] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:00.085779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b389129-7c55-4177-8e96-d9eae11dfb4d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:00.089968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6b389129-7c55-4177-8e96-d9eae11dfb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:00.103852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2a09e-fb83-4b30-910e-a40e7a490dee None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:00.105415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5ba2a09e-fb83-4b30-910e-a40e7a490dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:00.180629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b389129-7c55-4177-8e96-d9eae11dfb4d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:00.190229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a505f16-5249-4b80-80c0-70e8bb915f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:00.190934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a505f16-5249-4b80-80c0-70e8bb915f04 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:00.191136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a505f16-5249-4b80-80c0-70e8bb915f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:00.191446 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a505f16-5249-4b80-80c0-70e8bb915f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:00.196259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2a09e-fb83-4b30-910e-a40e7a490dee None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:00.200455 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80239c2b-4d6c-4033-8e31-a077533d7ae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:00.203418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80239c2b-4d6c-4033-8e31-a077533d7ae9 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:00.203418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80239c2b-4d6c-4033-8e31-a077533d7ae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:00.203639 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80239c2b-4d6c-4033-8e31-a077533d7ae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:00.221430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80239c2b-4d6c-4033-8e31-a077533d7ae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:00.221993 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a505f16-5249-4b80-80c0-70e8bb915f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:00.223589 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a505f16-5249-4b80-80c0-70e8bb915f04 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:00.223699 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80239c2b-4d6c-4033-8e31-a077533d7ae9 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:00.237702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 950/1904] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:00.247090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 955/1905] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:00.884918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a505f16-5249-4b80-80c0-70e8bb915f04 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:00.886549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6a505f16-5249-4b80-80c0-70e8bb915f04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:00.908394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a505f16-5249-4b80-80c0-70e8bb915f04 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:00.924804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:00.925471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:00.925743 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:00.926011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] PATH_INFO: `/v3/projects/745d83dfccc54a0e8abb65c6e71e6923` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:00.927168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=745d83dfccc54a0e8abb65c6e71e6923)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:00.929122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:01.066798 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '745d83dfccc54a0e8abb65c6e71e6923', 'request_id': 'req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.085875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '745d83dfccc54a0e8abb65c6e71e6923', 'request_id': 'req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.096272 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '745d83dfccc54a0e8abb65c6e71e6923', 'request_id': 'req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.112175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80239c2b-4d6c-4033-8e31-a077533d7ae9 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:01.113977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-80239c2b-4d6c-4033-8e31-a077533d7ae9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:01.114268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] The token cache is being invalidate because project 745d83dfccc54a0e8abb65c6e71e6923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:01.114958 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.117283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.119679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 951/1906] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:00 2022] DELETE /identity/v3/projects/745d83dfccc54a0e8abb65c6e71e6923 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:01.123579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:01.123720 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:01.131830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:01.138381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80239c2b-4d6c-4033-8e31-a077533d7ae9 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:01.169879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:01.277078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9b7bc-60eb-4e9e-959f-befbaf3598cb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:01.293663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:01.295028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:01.295754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:01.298212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:01.298755 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] PATH_INFO: `/v3/projects/31afe21d75ba4238bf50a2f6a4e1b948` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:01.300143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:01.301028 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:01.301504 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] RBAC: Authorizing `identity:delete_project(project_id=31afe21d75ba4238bf50a2f6a4e1b948)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:01.302333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] PATH_INFO: `/v3/users/4694c1d16ba547edb447d8f2d763a42f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:01.307358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4694c1d16ba547edb447d8f2d763a42f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:01.308264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:01.311312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:01.417360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31afe21d75ba4238bf50a2f6a4e1b948', 'request_id': 'req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.425270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4694c1d16ba547edb447d8f2d763a42f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.426230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4694c1d16ba547edb447d8f2d763a42f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.434722 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31afe21d75ba4238bf50a2f6a4e1b948', 'request_id': 'req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.437854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4694c1d16ba547edb447d8f2d763a42f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.443934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31afe21d75ba4238bf50a2f6a4e1b948', 'request_id': 'req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.461957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4694c1d16ba547edb447d8f2d763a42f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.462844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4694c1d16ba547edb447d8f2d763a42f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.471180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] The token cache is being invalidate because project 31afe21d75ba4238bf50a2f6a4e1b948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:01.471794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.473439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.477411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 956/1907] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:01 2022] DELETE /identity/v3/projects/31afe21d75ba4238bf50a2f6a4e1b948 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:19:01.481413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:01.485724 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 952/1908] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:19:01 2022] DELETE /identity/v3/users/4694c1d16ba547edb447d8f2d763a42f => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:01.487411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:01.610948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a93c6d-7207-40ef-8c0f-742cbb2fa59d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:01.670598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:01.740690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65930e77-1283-443c-b4bf-2c2f07d6c756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:01.740690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65930e77-1283-443c-b4bf-2c2f07d6c756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:01.740690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65930e77-1283-443c-b4bf-2c2f07d6c756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:01.740690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65930e77-1283-443c-b4bf-2c2f07d6c756 admin admin] PATH_INFO: `/v3/users/18680d388391427db7e3e1d95774004d` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:01.741664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65930e77-1283-443c-b4bf-2c2f07d6c756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18680d388391427db7e3e1d95774004d)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:01.741664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65930e77-1283-443c-b4bf-2c2f07d6c756 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:01.857813 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-65930e77-1283-443c-b4bf-2c2f07d6c756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18680d388391427db7e3e1d95774004d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.872337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-65930e77-1283-443c-b4bf-2c2f07d6c756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18680d388391427db7e3e1d95774004d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.887093 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-65930e77-1283-443c-b4bf-2c2f07d6c756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18680d388391427db7e3e1d95774004d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.902045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-65930e77-1283-443c-b4bf-2c2f07d6c756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18680d388391427db7e3e1d95774004d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.902923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-65930e77-1283-443c-b4bf-2c2f07d6c756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18680d388391427db7e3e1d95774004d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:01.914288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 957/1909] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:19:01 2022] DELETE /identity/v3/users/18680d388391427db7e3e1d95774004d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:02.089705 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:02.091818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:02.111021 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48bf97-5623-4ec7-be08-2e2bc0ec7baa admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:02.222325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:02.224263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.224879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.225186 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] PATH_INFO: `/v3/projects/624ab6a265f24ea08f75645d45230b9b` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.226849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] RBAC: Authorizing `identity:delete_project(project_id=624ab6a265f24ea08f75645d45230b9b)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:02.229160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:02.305474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:02.312577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624ab6a265f24ea08f75645d45230b9b', 'request_id': 'req-259f3de2-31fd-43e2-88e4-2e132ec7c44e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:02.325216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624ab6a265f24ea08f75645d45230b9b', 'request_id': 'req-259f3de2-31fd-43e2-88e4-2e132ec7c44e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:02.333720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624ab6a265f24ea08f75645d45230b9b', 'request_id': 'req-259f3de2-31fd-43e2-88e4-2e132ec7c44e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:02.359404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] The token cache is being invalidate because project 624ab6a265f24ea08f75645d45230b9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:02.359784 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:02.360507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:02.362237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 953/1910] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:02 2022] DELETE /identity/v3/projects/624ab6a265f24ea08f75645d45230b9b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:02.369731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add72c9-cf79-46fb-8d4b-2b3aa8a34780 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.370207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add72c9-cf79-46fb-8d4b-2b3aa8a34780 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.370342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add72c9-cf79-46fb-8d4b-2b3aa8a34780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.395780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1add72c9-cf79-46fb-8d4b-2b3aa8a34780 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:02.397216 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1add72c9-cf79-46fb-8d4b-2b3aa8a34780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:02.398667 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:02.400214 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:02.467290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f3de2-31fd-43e2-88e4-2e132ec7c44e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:02.472604 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b43496-2928-467a-9166-4d60daec2795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:02.473332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b43496-2928-467a-9166-4d60daec2795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.473706 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b43496-2928-467a-9166-4d60daec2795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.474072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b43496-2928-467a-9166-4d60daec2795 admin admin] PATH_INFO: `/v3/projects/c6a818815c7c404eb86bb41e1cde04b7` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.475304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b43496-2928-467a-9166-4d60daec2795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a818815c7c404eb86bb41e1cde04b7)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:02.478060 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b43496-2928-467a-9166-4d60daec2795 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:02.479951 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 958/1911] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:02.488957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a3cfd-b34b-4b80-a083-68b958a5063b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.489231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a3cfd-b34b-4b80-a083-68b958a5063b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.489560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a3cfd-b34b-4b80-a083-68b958a5063b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.512177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7f7a3cfd-b34b-4b80-a083-68b958a5063b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:02.513630 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7f7a3cfd-b34b-4b80-a083-68b958a5063b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:02.526230 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 959/1912] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:19:02.531229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a3cfd-b34b-4b80-a083-68b958a5063b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:02.539965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a3cfd-b34b-4b80-a083-68b958a5063b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:02.542989 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c4b43496-2928-467a-9166-4d60daec2795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a818815c7c404eb86bb41e1cde04b7', 'request_id': 'req-c4b43496-2928-467a-9166-4d60daec2795'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:02.554195 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c4b43496-2928-467a-9166-4d60daec2795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a818815c7c404eb86bb41e1cde04b7', 'request_id': 'req-c4b43496-2928-467a-9166-4d60daec2795'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:02.562313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c4b43496-2928-467a-9166-4d60daec2795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a818815c7c404eb86bb41e1cde04b7', 'request_id': 'req-c4b43496-2928-467a-9166-4d60daec2795'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:02.581833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c4b43496-2928-467a-9166-4d60daec2795 admin admin] The token cache is being invalidate because project c6a818815c7c404eb86bb41e1cde04b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:02.582191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c4b43496-2928-467a-9166-4d60daec2795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:02.583036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c4b43496-2928-467a-9166-4d60daec2795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:02.588664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 954/1913] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:02 2022] DELETE /identity/v3/projects/c6a818815c7c404eb86bb41e1cde04b7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:19:02.590230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c2319-3a8c-4638-977f-9076d5f128e8 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.590873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c2319-3a8c-4638-977f-9076d5f128e8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.591228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c2319-3a8c-4638-977f-9076d5f128e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.593520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:02.604713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b192e7e-2d08-4e02-b0c6-f57a0346242e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:02.605331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b192e7e-2d08-4e02-b0c6-f57a0346242e admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.605614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b192e7e-2d08-4e02-b0c6-f57a0346242e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.605937 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b192e7e-2d08-4e02-b0c6-f57a0346242e admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.606547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b192e7e-2d08-4e02-b0c6-f57a0346242e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:02.606837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b192e7e-2d08-4e02-b0c6-f57a0346242e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:02.608457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b192e7e-2d08-4e02-b0c6-f57a0346242e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:02.612191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-375c2319-3a8c-4638-977f-9076d5f128e8 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:02.613803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-375c2319-3a8c-4638-977f-9076d5f128e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:02.624665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 960/1914] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:19:02.631640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 955/1915] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:02.631798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b192e7e-2d08-4e02-b0c6-f57a0346242e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:02.631798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7b192e7e-2d08-4e02-b0c6-f57a0346242e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:02.633221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9706a75-cb35-4e65-933d-c42504d2af3a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.633656 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9706a75-cb35-4e65-933d-c42504d2af3a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.634170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9706a75-cb35-4e65-933d-c42504d2af3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.641192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b192e7e-2d08-4e02-b0c6-f57a0346242e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:02.645661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183d8cf-4096-4ed8-816f-482737ed7eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:02.646270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183d8cf-4096-4ed8-816f-482737ed7eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.646526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183d8cf-4096-4ed8-816f-482737ed7eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.646787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183d8cf-4096-4ed8-816f-482737ed7eb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.647408 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183d8cf-4096-4ed8-816f-482737ed7eb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:02.649334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e183d8cf-4096-4ed8-816f-482737ed7eb2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:02.654660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f9706a75-cb35-4e65-933d-c42504d2af3a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:02.655926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f9706a75-cb35-4e65-933d-c42504d2af3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:02.663872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 956/1916] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:19:02.668332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9706a75-cb35-4e65-933d-c42504d2af3a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:02.677148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9706a75-cb35-4e65-933d-c42504d2af3a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:02.681677 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 961/1917] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:02 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:02.681835 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dba9ec-1021-44a0-ad0a-cc579fe8c1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:02.682393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dba9ec-1021-44a0-ad0a-cc579fe8c1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.682661 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dba9ec-1021-44a0-ad0a-cc579fe8c1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.682920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dba9ec-1021-44a0-ad0a-cc579fe8c1d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.683561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dba9ec-1021-44a0-ad0a-cc579fe8c1d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:02.683857 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dba9ec-1021-44a0-ad0a-cc579fe8c1d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:02.685461 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183d8cf-4096-4ed8-816f-482737ed7eb2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:02.685596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dba9ec-1021-44a0-ad0a-cc579fe8c1d4 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:02.701240 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 957/1918] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:19:02.702225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dba9ec-1021-44a0-ad0a-cc579fe8c1d4 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:02.713476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dba9ec-1021-44a0-ad0a-cc579fe8c1d4 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:02.748372 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:02.755434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183d8cf-4096-4ed8-816f-482737ed7eb2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:02.761190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5495b4-09fe-4a57-9f86-d966eac28fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:02.762222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5495b4-09fe-4a57-9f86-d966eac28fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.762496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5495b4-09fe-4a57-9f86-d966eac28fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.762764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5495b4-09fe-4a57-9f86-d966eac28fc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.763745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5495b4-09fe-4a57-9f86-d966eac28fc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:02.766003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5495b4-09fe-4a57-9f86-d966eac28fc2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:02.778692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b64263-9490-4a4e-aab7-c8b4751a4e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:02.779183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b64263-9490-4a4e-aab7-c8b4751a4e9c admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.779460 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b64263-9490-4a4e-aab7-c8b4751a4e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.779728 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b64263-9490-4a4e-aab7-c8b4751a4e9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.780380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b64263-9490-4a4e-aab7-c8b4751a4e9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:02.782331 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b64263-9490-4a4e-aab7-c8b4751a4e9c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:02.790418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 962/1919] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:02 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:19:02.794253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5495b4-09fe-4a57-9f86-d966eac28fc2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:02.804572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5495b4-09fe-4a57-9f86-d966eac28fc2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:02.811718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae550e-6523-495b-a824-3cde4d07b59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:02.815635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 958/1920] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:02 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:02.815783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae550e-6523-495b-a824-3cde4d07b59e admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.815783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae550e-6523-495b-a824-3cde4d07b59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.815783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae550e-6523-495b-a824-3cde4d07b59e admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.815783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae550e-6523-495b-a824-3cde4d07b59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:02.816543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae550e-6523-495b-a824-3cde4d07b59e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:02.816956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b64263-9490-4a4e-aab7-c8b4751a4e9c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:02.830268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 963/1921] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:19:02.834398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae550e-6523-495b-a824-3cde4d07b59e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:02.846368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae550e-6523-495b-a824-3cde4d07b59e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:02.929579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b64263-9490-4a4e-aab7-c8b4751a4e9c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:02.938866 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3dda50-86eb-4cb5-a075-1ddca992f767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:02.939491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3dda50-86eb-4cb5-a075-1ddca992f767 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.939757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3dda50-86eb-4cb5-a075-1ddca992f767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.940024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3dda50-86eb-4cb5-a075-1ddca992f767 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.940699 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3dda50-86eb-4cb5-a075-1ddca992f767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:02.942626 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3dda50-86eb-4cb5-a075-1ddca992f767 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:02.965883 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc0616-066a-46c6-8b29-f01683f515e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:02.966612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc0616-066a-46c6-8b29-f01683f515e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.966948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc0616-066a-46c6-8b29-f01683f515e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.967309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc0616-066a-46c6-8b29-f01683f515e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.967822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 959/1922] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:02 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:02.968100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc0616-066a-46c6-8b29-f01683f515e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:02.970069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc0616-066a-46c6-8b29-f01683f515e0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:02.972256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3dda50-86eb-4cb5-a075-1ddca992f767 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:02.980737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 964/1923] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:02.984660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc0616-066a-46c6-8b29-f01683f515e0 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:02.987278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3dda50-86eb-4cb5-a075-1ddca992f767 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:02.992727 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f2649-bfc4-4d24-a635-2a67d89593b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:02.993398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f2649-bfc4-4d24-a635-2a67d89593b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:02.993742 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f2649-bfc4-4d24-a635-2a67d89593b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:02.994180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f2649-bfc4-4d24-a635-2a67d89593b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:02.994931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f2649-bfc4-4d24-a635-2a67d89593b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:02.996112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc0616-066a-46c6-8b29-f01683f515e0 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:02.996712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40f2649-bfc4-4d24-a635-2a67d89593b9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:03.001149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b8a0e-51ba-4c86-94fb-27a998128cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:03.001935 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b8a0e-51ba-4c86-94fb-27a998128cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:03.002294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b8a0e-51ba-4c86-94fb-27a998128cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:03.002626 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0b8a0e-51ba-4c86-94fb-27a998128cc3 admin admin] PATH_INFO: `/v3/projects/e161ec22343e45a8b0ace2c6611d1bf1/users/5d641b3307e64a11b6cd4f18f02248d2/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:03.009125 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 960/1924] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:03.012848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f2649-bfc4-4d24-a635-2a67d89593b9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:03.020472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b8a0e-51ba-4c86-94fb-27a998128cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e161ec22343e45a8b0ace2c6611d1bf1, user_id=5d641b3307e64a11b6cd4f18f02248d2, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:03.023148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f2649-bfc4-4d24-a635-2a67d89593b9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:03.023262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0b8a0e-51ba-4c86-94fb-27a998128cc3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:03.028356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4658e9-976c-4ffb-bb39-13b1dd8f5fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:03.029122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4658e9-976c-4ffb-bb39-13b1dd8f5fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:03.029350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4658e9-976c-4ffb-bb39-13b1dd8f5fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:03.029715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4658e9-976c-4ffb-bb39-13b1dd8f5fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:03.030495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4658e9-976c-4ffb-bb39-13b1dd8f5fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:03.032144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4658e9-976c-4ffb-bb39-13b1dd8f5fb1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:03.052734 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 965/1925] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:02 2022] PUT /identity/v3/projects/e161ec22343e45a8b0ace2c6611d1bf1/users/5d641b3307e64a11b6cd4f18f02248d2/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:19:03.053133 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 961/1926] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:19:03.053888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b8a0e-51ba-4c86-94fb-27a998128cc3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:03.056485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b125b-fba0-4fd3-b17f-83106afdd1f8 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:03.056821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b125b-fba0-4fd3-b17f-83106afdd1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:03.057093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b125b-fba0-4fd3-b17f-83106afdd1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:03.077696 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:03.098605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-046b125b-fba0-4fd3-b17f-83106afdd1f8 None None] MFA Rules not processed for user `5d641b3307e64a11b6cd4f18f02248d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:03.099825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-046b125b-fba0-4fd3-b17f-83106afdd1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:03.147471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0b8a0e-51ba-4c86-94fb-27a998128cc3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:03.156192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5260d2-5db3-4502-990d-199199853d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:03.157981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5260d2-5db3-4502-990d-199199853d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:03.158501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5260d2-5db3-4502-990d-199199853d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:03.158874 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5260d2-5db3-4502-990d-199199853d2a admin admin] PATH_INFO: `/v3/projects/0d32aae92a1f4c198d657e7fd0b8551c/users/f9308e0c9c1a45d8a088278c282dec6e/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:03.179592 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5260d2-5db3-4502-990d-199199853d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d32aae92a1f4c198d657e7fd0b8551c, user_id=f9308e0c9c1a45d8a088278c282dec6e, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:03.183330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5260d2-5db3-4502-990d-199199853d2a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:03.202001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 966/1927] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:03 2022] PUT /identity/v3/projects/0d32aae92a1f4c198d657e7fd0b8551c/users/f9308e0c9c1a45d8a088278c282dec6e/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:03.210625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 962/1928] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:03.213614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ce683-599b-4d44-88cd-115a55e10976 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:03.214045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ce683-599b-4d44-88cd-115a55e10976 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:03.214511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ce683-599b-4d44-88cd-115a55e10976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:03.230277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b125b-fba0-4fd3-b17f-83106afdd1f8 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:03.231932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-046b125b-fba0-4fd3-b17f-83106afdd1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:03.260955 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-aa7ce683-599b-4d44-88cd-115a55e10976 None None] MFA Rules not processed for user `f9308e0c9c1a45d8a088278c282dec6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:03.262307 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-aa7ce683-599b-4d44-88cd-115a55e10976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:03.264080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:03.311615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b125b-fba0-4fd3-b17f-83106afdd1f8 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:03.337457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 967/1929] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:03.338942 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ce683-599b-4d44-88cd-115a55e10976 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:03.377423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37894862-753b-412f-a855-f0e8490cf29d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac783fbe95ea4b74bd46af415c9734fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:03.381092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37894862-753b-412f-a855-f0e8490cf29d service glance] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:03.381370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37894862-753b-412f-a855-f0e8490cf29d service glance] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:03.383285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37894862-753b-412f-a855-f0e8490cf29d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:03.454378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ce683-599b-4d44-88cd-115a55e10976 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:03.460153 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f7955-c4c8-49f5-a826-2efbae785ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:03.460870 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f7955-c4c8-49f5-a826-2efbae785ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:03.461299 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f7955-c4c8-49f5-a826-2efbae785ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:03.461743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f7955-c4c8-49f5-a826-2efbae785ac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:03.462592 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f7955-c4c8-49f5-a826-2efbae785ac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:03.464793 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f7955-c4c8-49f5-a826-2efbae785ac6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:03.478540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37894862-753b-412f-a855-f0e8490cf29d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:03.480015 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37894862-753b-412f-a855-f0e8490cf29d service glance] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:03.490343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 963/1930] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:19:03.506565 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 968/1931] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:03 2022] POST /identity/v3/projects => generated 387 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:19:03.507284 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37894862-753b-412f-a855-f0e8490cf29d service glance] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:03.523375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37894862-753b-412f-a855-f0e8490cf29d service glance] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:03.628906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126c5c3-e246-47b7-8218-6b3c547e219e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:03.629565 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126c5c3-e246-47b7-8218-6b3c547e219e admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:03.629835 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126c5c3-e246-47b7-8218-6b3c547e219e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:03.630101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126c5c3-e246-47b7-8218-6b3c547e219e admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:03.630811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126c5c3-e246-47b7-8218-6b3c547e219e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:03.632993 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126c5c3-e246-47b7-8218-6b3c547e219e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:03.667078 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f7955-c4c8-49f5-a826-2efbae785ac6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:03.668667 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 964/1932] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:03 2022] POST /identity/v3/users => generated 300 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:03.679506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f7955-c4c8-49f5-a826-2efbae785ac6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:03.706937 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96709cc1-f908-4641-bf07-983336de6e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:03.709624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96709cc1-f908-4641-bf07-983336de6e3b admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:03.710622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96709cc1-f908-4641-bf07-983336de6e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:03.712025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96709cc1-f908-4641-bf07-983336de6e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:03.714939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96709cc1-f908-4641-bf07-983336de6e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:03.717620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96709cc1-f908-4641-bf07-983336de6e3b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:03.754705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 969/1933] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:03.759181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126c5c3-e246-47b7-8218-6b3c547e219e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:03.774430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126c5c3-e246-47b7-8218-6b3c547e219e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:03.780505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1390563-bf51-400f-8266-92cd451168b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:03.781217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1390563-bf51-400f-8266-92cd451168b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:03.781491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1390563-bf51-400f-8266-92cd451168b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:03.781756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1390563-bf51-400f-8266-92cd451168b2 admin admin] PATH_INFO: `/v3/projects/582beb485ae043fe9c2b63d0d35dfe09/users/74a66959d2ca4e4ead9baf4975a3200d/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:03.807634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1390563-bf51-400f-8266-92cd451168b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=582beb485ae043fe9c2b63d0d35dfe09, user_id=74a66959d2ca4e4ead9baf4975a3200d, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:03.811482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1390563-bf51-400f-8266-92cd451168b2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:03.815755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:03.845047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 965/1934] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:03 2022] PUT /identity/v3/projects/582beb485ae043fe9c2b63d0d35dfe09/users/74a66959d2ca4e4ead9baf4975a3200d/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:03.846344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96709cc1-f908-4641-bf07-983336de6e3b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:03.853361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:03.859960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96709cc1-f908-4641-bf07-983336de6e3b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:03.952179 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f75bc28-9b17-491a-9559-5ddfc11a5bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:03.953032 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75bc28-9b17-491a-9559-5ddfc11a5bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:03.955222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75bc28-9b17-491a-9559-5ddfc11a5bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:03.955222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75bc28-9b17-491a-9559-5ddfc11a5bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:03.955741 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f75bc28-9b17-491a-9559-5ddfc11a5bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:03.957928 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f75bc28-9b17-491a-9559-5ddfc11a5bf6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:03.970924 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 970/1935] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:19:03.975398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1390563-bf51-400f-8266-92cd451168b2 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:03.990990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1390563-bf51-400f-8266-92cd451168b2 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:03.997464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f23b68-b541-4423-bf36-cadf4e988e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:03.998276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f23b68-b541-4423-bf36-cadf4e988e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:03.998641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f23b68-b541-4423-bf36-cadf4e988e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:03.998994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f23b68-b541-4423-bf36-cadf4e988e73 admin admin] PATH_INFO: `/v3/projects/582beb485ae043fe9c2b63d0d35dfe09/users/74a66959d2ca4e4ead9baf4975a3200d/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:04.001065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f23b68-b541-4423-bf36-cadf4e988e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=582beb485ae043fe9c2b63d0d35dfe09, user_id=74a66959d2ca4e4ead9baf4975a3200d, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:04.004169 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f23b68-b541-4423-bf36-cadf4e988e73 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:04.018576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 966/1936] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:03 2022] PUT /identity/v3/projects/582beb485ae043fe9c2b63d0d35dfe09/users/74a66959d2ca4e4ead9baf4975a3200d/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:19:04.029262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e7c25-c6c3-4e5f-ab59-5d4f59e37385 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:04.029903 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e7c25-c6c3-4e5f-ab59-5d4f59e37385 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:04.030074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e7c25-c6c3-4e5f-ab59-5d4f59e37385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:04.084930 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e10e7c25-c6c3-4e5f-ab59-5d4f59e37385 None None] MFA Rules not processed for user `74a66959d2ca4e4ead9baf4975a3200d`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:04.086683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e10e7c25-c6c3-4e5f-ab59-5d4f59e37385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:04.269213 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 971/1937] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:04.281398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f23b68-b541-4423-bf36-cadf4e988e73 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:04.283722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-96f23b68-b541-4423-bf36-cadf4e988e73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:04.361222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f23b68-b541-4423-bf36-cadf4e988e73 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:04.365250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c3c70-5d0c-4848-939d-1e8d5e18f26c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:04.365951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c3c70-5d0c-4848-939d-1e8d5e18f26c service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:04.366217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c3c70-5d0c-4848-939d-1e8d5e18f26c service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:04.366479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c3c70-5d0c-4848-939d-1e8d5e18f26c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:04.376088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c3c70-5d0c-4848-939d-1e8d5e18f26c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:04.377555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c3c70-5d0c-4848-939d-1e8d5e18f26c service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:04.387230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 967/1938] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:04.470338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e7c25-c6c3-4e5f-ab59-5d4f59e37385 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:04.544508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e7c25-c6c3-4e5f-ab59-5d4f59e37385 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:04.549942 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:04.550585 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:04.550855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:04.551117 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] PATH_INFO: `/v3/users/f9308e0c9c1a45d8a088278c282dec6e` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:04.552350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9308e0c9c1a45d8a088278c282dec6e)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:04.554318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:04.585780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:04.607293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9308e0c9c1a45d8a088278c282dec6e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:04.616180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9308e0c9c1a45d8a088278c282dec6e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:04.630817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9308e0c9c1a45d8a088278c282dec6e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:04.654656 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9308e0c9c1a45d8a088278c282dec6e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:04.655380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9308e0c9c1a45d8a088278c282dec6e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:04.663858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 972/1939] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:19:04 2022] DELETE /identity/v3/users/f9308e0c9c1a45d8a088278c282dec6e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:04.670930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a2116-a688-42f0-96bf-e30b9655ada5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:04.672434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a2116-a688-42f0-96bf-e30b9655ada5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:04.672693 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a2116-a688-42f0-96bf-e30b9655ada5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:04.692796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-cb9a2116-a688-42f0-96bf-e30b9655ada5 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:04.694017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-cb9a2116-a688-42f0-96bf-e30b9655ada5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:04.765472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 968/1940] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:04.788634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:04.791039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:04.852694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646d12e-2a13-4d39-80e2-218dc4717983 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:04.856269 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61db4e-5cfc-457a-b507-d74c1c086b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:04.856921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61db4e-5cfc-457a-b507-d74c1c086b0a service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:04.857194 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61db4e-5cfc-457a-b507-d74c1c086b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:04.857460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61db4e-5cfc-457a-b507-d74c1c086b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:04.866656 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61db4e-5cfc-457a-b507-d74c1c086b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:04.868742 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d61db4e-5cfc-457a-b507-d74c1c086b0a service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:04.878067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 973/1941] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:05.317404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a2116-a688-42f0-96bf-e30b9655ada5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:05.321759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-cb9a2116-a688-42f0-96bf-e30b9655ada5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:05.336358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a2116-a688-42f0-96bf-e30b9655ada5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:05.352631 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:05.356309 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:05.356890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:05.357377 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] PATH_INFO: `/v3/projects/0d32aae92a1f4c198d657e7fd0b8551c` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:05.359115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d32aae92a1f4c198d657e7fd0b8551c)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:05.387086 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:05.455696 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d32aae92a1f4c198d657e7fd0b8551c', 'request_id': 'req-31baee69-ef78-40d3-b452-208ca1daaf2d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:05.465994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d32aae92a1f4c198d657e7fd0b8551c', 'request_id': 'req-31baee69-ef78-40d3-b452-208ca1daaf2d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:05.474919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d32aae92a1f4c198d657e7fd0b8551c', 'request_id': 'req-31baee69-ef78-40d3-b452-208ca1daaf2d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:05.489721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] The token cache is being invalidate because project 0d32aae92a1f4c198d657e7fd0b8551c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:05.490008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:05.490758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:05.492431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 969/1942] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:05 2022] DELETE /identity/v3/projects/0d32aae92a1f4c198d657e7fd0b8551c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:05.497215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61db4e-5cfc-457a-b507-d74c1c086b0a service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:05.541799 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61db4e-5cfc-457a-b507-d74c1c086b0a service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:05.610104 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f409d4-d6a7-4ccd-ac78-ae5cd7860424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:05.610743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f409d4-d6a7-4ccd-ac78-ae5cd7860424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:05.611017 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f409d4-d6a7-4ccd-ac78-ae5cd7860424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:05.611284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f409d4-d6a7-4ccd-ac78-ae5cd7860424 admin admin] PATH_INFO: `/v3/users/74a66959d2ca4e4ead9baf4975a3200d` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:05.612475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f409d4-d6a7-4ccd-ac78-ae5cd7860424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a66959d2ca4e4ead9baf4975a3200d)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:05.614813 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f409d4-d6a7-4ccd-ac78-ae5cd7860424 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:05.667483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-48f409d4-d6a7-4ccd-ac78-ae5cd7860424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a66959d2ca4e4ead9baf4975a3200d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:05.678969 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:05.705394 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-48f409d4-d6a7-4ccd-ac78-ae5cd7860424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a66959d2ca4e4ead9baf4975a3200d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:05.730209 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-48f409d4-d6a7-4ccd-ac78-ae5cd7860424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a66959d2ca4e4ead9baf4975a3200d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:05.745062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-48f409d4-d6a7-4ccd-ac78-ae5cd7860424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a66959d2ca4e4ead9baf4975a3200d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:05.745921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-48f409d4-d6a7-4ccd-ac78-ae5cd7860424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a66959d2ca4e4ead9baf4975a3200d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:05.755325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 974/1943] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:19:05 2022] DELETE /identity/v3/users/74a66959d2ca4e4ead9baf4975a3200d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:06.401023 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:06.402598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:06.417387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31baee69-ef78-40d3-b452-208ca1daaf2d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:06.422668 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:06.501548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:06.502209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:06.502482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:06.502750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717 admin admin] PATH_INFO: `/v3/projects/582beb485ae043fe9c2b63d0d35dfe09` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:06.503875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=582beb485ae043fe9c2b63d0d35dfe09)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:06.505800 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:06.558017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '582beb485ae043fe9c2b63d0d35dfe09', 'request_id': 'req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:06.574260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '582beb485ae043fe9c2b63d0d35dfe09', 'request_id': 'req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:06.582237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '582beb485ae043fe9c2b63d0d35dfe09', 'request_id': 'req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:06.597120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717 admin admin] The token cache is being invalidate because project 582beb485ae043fe9c2b63d0d35dfe09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:06.597680 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:06.598677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bc7ccc3d-584a-45b4-a4dc-8d8d6a540717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:06.600395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 970/1944] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:06 2022] DELETE /identity/v3/projects/582beb485ae043fe9c2b63d0d35dfe09 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:06.605332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0908f8-ae36-4a4d-ae95-26b847a2d646 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:06.605839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0908f8-ae36-4a4d-ae95-26b847a2d646 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:06.605914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0908f8-ae36-4a4d-ae95-26b847a2d646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:06.630753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7b0908f8-ae36-4a4d-ae95-26b847a2d646 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:06.632083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7b0908f8-ae36-4a4d-ae95-26b847a2d646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:06.694610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 975/1945] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:06.702927 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e3d0d-ec70-4503-9a60-454223f9d3d4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:06.703205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e3d0d-ec70-4503-9a60-454223f9d3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:06.703473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e3d0d-ec70-4503-9a60-454223f9d3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:06.724217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5c6e3d0d-ec70-4503-9a60-454223f9d3d4 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:06.725519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5c6e3d0d-ec70-4503-9a60-454223f9d3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:06.733425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 971/1946] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:06.737811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0908f8-ae36-4a4d-ae95-26b847a2d646 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:06.748218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0908f8-ae36-4a4d-ae95-26b847a2d646 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:06.753224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b908fc6-1871-4453-b46f-85edb1c0c8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:06.753839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b908fc6-1871-4453-b46f-85edb1c0c8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:06.754123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b908fc6-1871-4453-b46f-85edb1c0c8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:06.754527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b908fc6-1871-4453-b46f-85edb1c0c8e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:06.755163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b908fc6-1871-4453-b46f-85edb1c0c8e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:06.755360 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b908fc6-1871-4453-b46f-85edb1c0c8e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:06.756990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b908fc6-1871-4453-b46f-85edb1c0c8e6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:06.772394 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 976/1947] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:19:06.776252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e3d0d-ec70-4503-9a60-454223f9d3d4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:06.785865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e3d0d-ec70-4503-9a60-454223f9d3d4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:06.790677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa86ea6-35a5-4b4c-92d1-7a8cce29e562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:06.791285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa86ea6-35a5-4b4c-92d1-7a8cce29e562 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:06.791557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa86ea6-35a5-4b4c-92d1-7a8cce29e562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:06.791823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa86ea6-35a5-4b4c-92d1-7a8cce29e562 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:06.792468 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa86ea6-35a5-4b4c-92d1-7a8cce29e562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:06.794519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa86ea6-35a5-4b4c-92d1-7a8cce29e562 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:06.828758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 972/1948] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:06 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:19:06.829864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b908fc6-1871-4453-b46f-85edb1c0c8e6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:06.841830 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b908fc6-1871-4453-b46f-85edb1c0c8e6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:06.902596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482debca-4740-400c-951f-c128298c84d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:06.903151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482debca-4740-400c-951f-c128298c84d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:06.903423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482debca-4740-400c-951f-c128298c84d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:06.903687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482debca-4740-400c-951f-c128298c84d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:06.904359 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482debca-4740-400c-951f-c128298c84d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:06.906309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482debca-4740-400c-951f-c128298c84d6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:06.932155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 977/1949] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:06 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:19:06.936232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa86ea6-35a5-4b4c-92d1-7a8cce29e562 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:06.946709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa86ea6-35a5-4b4c-92d1-7a8cce29e562 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:06.951367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15c038-e887-4de9-bc56-16baa358f88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:06.951964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15c038-e887-4de9-bc56-16baa358f88d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:06.952241 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15c038-e887-4de9-bc56-16baa358f88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:06.952530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15c038-e887-4de9-bc56-16baa358f88d admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:06.953211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15c038-e887-4de9-bc56-16baa358f88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:06.954808 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15c038-e887-4de9-bc56-16baa358f88d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:06.964778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 973/1950] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:06.968895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482debca-4740-400c-951f-c128298c84d6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:06.978913 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482debca-4740-400c-951f-c128298c84d6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:06.983444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5164aae-974e-446a-8660-49226898a81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:06.984039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5164aae-974e-446a-8660-49226898a81d admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:06.984314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5164aae-974e-446a-8660-49226898a81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:06.984615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5164aae-974e-446a-8660-49226898a81d admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:06.985261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5164aae-974e-446a-8660-49226898a81d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:06.986872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5164aae-974e-446a-8660-49226898a81d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:06.998237 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 978/1951] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:07.002306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15c038-e887-4de9-bc56-16baa358f88d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:07.012835 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15c038-e887-4de9-bc56-16baa358f88d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:07.017666 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175ccf5-32c8-470f-9580-28312eb6d528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:07.018438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175ccf5-32c8-470f-9580-28312eb6d528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:07.018621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175ccf5-32c8-470f-9580-28312eb6d528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:07.018886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d175ccf5-32c8-470f-9580-28312eb6d528 admin admin] PATH_INFO: `/v3/projects/251eaac1952247f9ad304278019eb7cf/users/9ddc4be0b0bf440baf9371be637e46bb/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:07.039285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175ccf5-32c8-470f-9580-28312eb6d528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=251eaac1952247f9ad304278019eb7cf, user_id=9ddc4be0b0bf440baf9371be637e46bb, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:07.042697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d175ccf5-32c8-470f-9580-28312eb6d528 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:07.057901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 974/1952] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:07 2022] PUT /identity/v3/projects/251eaac1952247f9ad304278019eb7cf/users/9ddc4be0b0bf440baf9371be637e46bb/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:07.069774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb88d2-db15-415f-9ad8-8d56c72d6501 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:07.070101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb88d2-db15-415f-9ad8-8d56c72d6501 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:07.070382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb88d2-db15-415f-9ad8-8d56c72d6501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:07.099774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-3ffb88d2-db15-415f-9ad8-8d56c72d6501 None None] MFA Rules not processed for user `9ddc4be0b0bf440baf9371be637e46bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:07.101054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3ffb88d2-db15-415f-9ad8-8d56c72d6501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:07.120939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:07.174491 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 979/1953] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:19:07.178842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175ccf5-32c8-470f-9580-28312eb6d528 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:07.219003 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:07.244220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d175ccf5-32c8-470f-9580-28312eb6d528 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:07.248985 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36d484-54b0-4ae9-b523-be6dfb9ed39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:07.249609 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36d484-54b0-4ae9-b523-be6dfb9ed39f admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:07.249884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36d484-54b0-4ae9-b523-be6dfb9ed39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:07.250126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36d484-54b0-4ae9-b523-be6dfb9ed39f admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:07.250773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36d484-54b0-4ae9-b523-be6dfb9ed39f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:07.252734 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36d484-54b0-4ae9-b523-be6dfb9ed39f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:07.287029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 975/1954] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:07 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:07.292529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb88d2-db15-415f-9ad8-8d56c72d6501 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:07.304296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb88d2-db15-415f-9ad8-8d56c72d6501 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:07.384296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb09fec-c9a0-4fbd-861a-3ee6203516b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:07.384943 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb09fec-c9a0-4fbd-861a-3ee6203516b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:07.385286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb09fec-c9a0-4fbd-861a-3ee6203516b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:07.385462 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb09fec-c9a0-4fbd-861a-3ee6203516b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:07.386097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb09fec-c9a0-4fbd-861a-3ee6203516b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:07.387970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb09fec-c9a0-4fbd-861a-3ee6203516b6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:07.416493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 980/1955] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:07 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:07.417720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36d484-54b0-4ae9-b523-be6dfb9ed39f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:07.428705 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36d484-54b0-4ae9-b523-be6dfb9ed39f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:07.446860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c03f5-52b0-4d7c-83aa-e31a10a93ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:07.447607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c03f5-52b0-4d7c-83aa-e31a10a93ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:07.447984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c03f5-52b0-4d7c-83aa-e31a10a93ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:07.448410 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c03f5-52b0-4d7c-83aa-e31a10a93ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:07.449245 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c03f5-52b0-4d7c-83aa-e31a10a93ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:07.450969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c03f5-52b0-4d7c-83aa-e31a10a93ab8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:07.463075 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 976/1956] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:07.467444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb09fec-c9a0-4fbd-861a-3ee6203516b6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:07.480274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb09fec-c9a0-4fbd-861a-3ee6203516b6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:07.486067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d62048-7ecb-411d-bc98-5df7ce0631f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:07.486864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d62048-7ecb-411d-bc98-5df7ce0631f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:07.487205 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d62048-7ecb-411d-bc98-5df7ce0631f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:07.487541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d62048-7ecb-411d-bc98-5df7ce0631f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:07.488735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d62048-7ecb-411d-bc98-5df7ce0631f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:07.490453 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d62048-7ecb-411d-bc98-5df7ce0631f2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:07.501898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 981/1957] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:07.506432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c03f5-52b0-4d7c-83aa-e31a10a93ab8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:07.521900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c03f5-52b0-4d7c-83aa-e31a10a93ab8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:07.529062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cfdb6-9191-46ff-b6a3-c5fe3c795ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:07.529795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cfdb6-9191-46ff-b6a3-c5fe3c795ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:07.530306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cfdb6-9191-46ff-b6a3-c5fe3c795ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:07.530428 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cfdb6-9191-46ff-b6a3-c5fe3c795ccc admin admin] PATH_INFO: `/v3/projects/8fa5357b890a4bcebc58466125d7ea2e/users/96dfb70bb97a4c32b68cc26769d94351/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:07.550030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cfdb6-9191-46ff-b6a3-c5fe3c795ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa5357b890a4bcebc58466125d7ea2e, user_id=96dfb70bb97a4c32b68cc26769d94351, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:07.552882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cfdb6-9191-46ff-b6a3-c5fe3c795ccc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:07.572690 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 977/1958] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:07 2022] PUT /identity/v3/projects/8fa5357b890a4bcebc58466125d7ea2e/users/96dfb70bb97a4c32b68cc26769d94351/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:07.583822 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dfa0e-ddd9-4801-a72c-0795ee2bc97b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:07.583956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dfa0e-ddd9-4801-a72c-0795ee2bc97b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:07.584160 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dfa0e-ddd9-4801-a72c-0795ee2bc97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:07.614614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-850dfa0e-ddd9-4801-a72c-0795ee2bc97b None None] MFA Rules not processed for user `96dfb70bb97a4c32b68cc26769d94351`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:07.615814 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-850dfa0e-ddd9-4801-a72c-0795ee2bc97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:07.691111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 982/1959] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:07.694999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cfdb6-9191-46ff-b6a3-c5fe3c795ccc admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:07.766015 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cfdb6-9191-46ff-b6a3-c5fe3c795ccc admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:07.772036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc75f67-2a69-4aa3-acda-a2b46a691203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:07.772685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc75f67-2a69-4aa3-acda-a2b46a691203 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:07.772966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc75f67-2a69-4aa3-acda-a2b46a691203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:07.773238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc75f67-2a69-4aa3-acda-a2b46a691203 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:07.774012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc75f67-2a69-4aa3-acda-a2b46a691203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:07.775950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc75f67-2a69-4aa3-acda-a2b46a691203 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:07.806551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 978/1960] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:07 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:07.810739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dfa0e-ddd9-4801-a72c-0795ee2bc97b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:07.827387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dfa0e-ddd9-4801-a72c-0795ee2bc97b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:07.858457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:07.896428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b43775-4b5c-4100-918c-a716114019f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:07.897090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b43775-4b5c-4100-918c-a716114019f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:07.897362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b43775-4b5c-4100-918c-a716114019f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:07.897651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b43775-4b5c-4100-918c-a716114019f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:07.898288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b43775-4b5c-4100-918c-a716114019f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:07.900625 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b43775-4b5c-4100-918c-a716114019f0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:07.931545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 983/1961] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:07 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:07.935528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc75f67-2a69-4aa3-acda-a2b46a691203 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:07.946732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc75f67-2a69-4aa3-acda-a2b46a691203 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:07.951300 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89706ad-65f0-4725-bd19-6eab2c58cfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:07.951911 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89706ad-65f0-4725-bd19-6eab2c58cfb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:07.952182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89706ad-65f0-4725-bd19-6eab2c58cfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:07.952446 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89706ad-65f0-4725-bd19-6eab2c58cfb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:07.953155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89706ad-65f0-4725-bd19-6eab2c58cfb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:07.954727 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89706ad-65f0-4725-bd19-6eab2c58cfb2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:07.966501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 979/1962] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:19:07.970831 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b43775-4b5c-4100-918c-a716114019f0 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:07.981761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b43775-4b5c-4100-918c-a716114019f0 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:07.986429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f405c5-be5d-44c6-beb6-89712970d61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:07.987267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f405c5-be5d-44c6-beb6-89712970d61d admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:07.987543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f405c5-be5d-44c6-beb6-89712970d61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:07.987812 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f405c5-be5d-44c6-beb6-89712970d61d admin admin] PATH_INFO: `/v3/projects/22aceb00f5194e7ab93ab70bcb9b9e27/users/573322e9d54d48838b90543b69c41527/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:08.003977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f405c5-be5d-44c6-beb6-89712970d61d admin admin] RBAC: Authorizing `identity:create_grant(project_id=22aceb00f5194e7ab93ab70bcb9b9e27, user_id=573322e9d54d48838b90543b69c41527, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:08.007483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f405c5-be5d-44c6-beb6-89712970d61d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:08.024784 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 984/1963] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:07 2022] PUT /identity/v3/projects/22aceb00f5194e7ab93ab70bcb9b9e27/users/573322e9d54d48838b90543b69c41527/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:08.029000 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89706ad-65f0-4725-bd19-6eab2c58cfb2 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:08.039900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89706ad-65f0-4725-bd19-6eab2c58cfb2 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:08.105139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968ec20-c9d7-409a-8685-04b7a3a68edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:08.105933 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968ec20-c9d7-409a-8685-04b7a3a68edd admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:08.106217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968ec20-c9d7-409a-8685-04b7a3a68edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:08.106484 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968ec20-c9d7-409a-8685-04b7a3a68edd admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:08.107154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968ec20-c9d7-409a-8685-04b7a3a68edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:08.108807 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2968ec20-c9d7-409a-8685-04b7a3a68edd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:08.111612 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:08.127345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 980/1964] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:08.132173 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f405c5-be5d-44c6-beb6-89712970d61d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:08.144046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f405c5-be5d-44c6-beb6-89712970d61d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:08.148866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6142fa3-473f-48e1-8e7b-ba1f598d6535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:08.149528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6142fa3-473f-48e1-8e7b-ba1f598d6535 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:08.149797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6142fa3-473f-48e1-8e7b-ba1f598d6535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:08.150059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6142fa3-473f-48e1-8e7b-ba1f598d6535 admin admin] PATH_INFO: `/v3/projects/22aceb00f5194e7ab93ab70bcb9b9e27/users/573322e9d54d48838b90543b69c41527/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:08.151940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6142fa3-473f-48e1-8e7b-ba1f598d6535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22aceb00f5194e7ab93ab70bcb9b9e27, user_id=573322e9d54d48838b90543b69c41527, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:08.154647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6142fa3-473f-48e1-8e7b-ba1f598d6535 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:08.166791 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 985/1965] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:08 2022] PUT /identity/v3/projects/22aceb00f5194e7ab93ab70bcb9b9e27/users/573322e9d54d48838b90543b69c41527/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:08.177587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319d376-2c89-48dd-9257-a07c45ff25d1 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:08.177901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319d376-2c89-48dd-9257-a07c45ff25d1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:08.178141 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319d376-2c89-48dd-9257-a07c45ff25d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:08.214929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2319d376-2c89-48dd-9257-a07c45ff25d1 None None] MFA Rules not processed for user `573322e9d54d48838b90543b69c41527`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:08.216166 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2319d376-2c89-48dd-9257-a07c45ff25d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:08.291545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 981/1966] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:08.303326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6142fa3-473f-48e1-8e7b-ba1f598d6535 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:08.306333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c6142fa3-473f-48e1-8e7b-ba1f598d6535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:08.381758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6142fa3-473f-48e1-8e7b-ba1f598d6535 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:08.390555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c6ac-e65b-4621-8047-cea45c415090 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:08.391199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c6ac-e65b-4621-8047-cea45c415090 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:08.391474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c6ac-e65b-4621-8047-cea45c415090 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:08.391738 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c6ac-e65b-4621-8047-cea45c415090 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:08.407296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15c6ac-e65b-4621-8047-cea45c415090 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:08.409480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15c6ac-e65b-4621-8047-cea45c415090 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:08.427756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 986/1967] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:09.298078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319d376-2c89-48dd-9257-a07c45ff25d1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:09.299593 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2319d376-2c89-48dd-9257-a07c45ff25d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:09.313020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319d376-2c89-48dd-9257-a07c45ff25d1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:09.316686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2883247-2368-4835-8c5d-355ca7f1cd8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:09.317347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2883247-2368-4835-8c5d-355ca7f1cd8a service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:09.317621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2883247-2368-4835-8c5d-355ca7f1cd8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:09.317887 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2883247-2368-4835-8c5d-355ca7f1cd8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:09.379544 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2883247-2368-4835-8c5d-355ca7f1cd8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:09.381024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2883247-2368-4835-8c5d-355ca7f1cd8a service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:09.390065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 982/1968] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:18.421505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=57,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:19.386104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=64 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:38.152977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c6ac-e65b-4621-8047-cea45c415090 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:38.162456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c6ac-e65b-4621-8047-cea45c415090 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:38.166357 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973b309-2600-4dd3-b5cf-cf1ef63c2821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:38.166958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973b309-2600-4dd3-b5cf-cf1ef63c2821 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:38.167233 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973b309-2600-4dd3-b5cf-cf1ef63c2821 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:38.167501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973b309-2600-4dd3-b5cf-cf1ef63c2821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:38.230177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973b309-2600-4dd3-b5cf-cf1ef63c2821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:38.231633 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973b309-2600-4dd3-b5cf-cf1ef63c2821 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:38.241534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 987/1969] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:38 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:41.233887 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2883247-2368-4835-8c5d-355ca7f1cd8a service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:41.375675 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2883247-2368-4835-8c5d-355ca7f1cd8a service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:41.381642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:41.382474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:41.382870 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:41.383302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] PATH_INFO: `/v3/users/9ddc4be0b0bf440baf9371be637e46bb` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:41.384781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ddc4be0b0bf440baf9371be637e46bb)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:41.387478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:41.453739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ddc4be0b0bf440baf9371be637e46bb'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:41.454714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ddc4be0b0bf440baf9371be637e46bb'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:41.463949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ddc4be0b0bf440baf9371be637e46bb'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:41.495232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ddc4be0b0bf440baf9371be637e46bb'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:41.496251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ddc4be0b0bf440baf9371be637e46bb'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:41.539970 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 983/1970] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:19:41 2022] DELETE /identity/v3/users/9ddc4be0b0bf440baf9371be637e46bb => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:41.549407 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a7cb9-a42b-4a8a-8102-939d640475cd None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:41.549861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a7cb9-a42b-4a8a-8102-939d640475cd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:41.550047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a7cb9-a42b-4a8a-8102-939d640475cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:41.579661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7c7a7cb9-a42b-4a8a-8102-939d640475cd None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:41.581162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7c7a7cb9-a42b-4a8a-8102-939d640475cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:41.647420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 988/1971] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:41.657298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:41.658780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:41.721343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb22c9-4fde-431a-a773-671c6408df26 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:41.725409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee10fa-c638-48a5-810c-75b55544af23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:41.726035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee10fa-c638-48a5-810c-75b55544af23 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:41.726311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee10fa-c638-48a5-810c-75b55544af23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:41.726580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee10fa-c638-48a5-810c-75b55544af23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:41.737699 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee10fa-c638-48a5-810c-75b55544af23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:41.739508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee10fa-c638-48a5-810c-75b55544af23 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:41.751392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 984/1972] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:42.075658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a7cb9-a42b-4a8a-8102-939d640475cd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:42.077860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7c7a7cb9-a42b-4a8a-8102-939d640475cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:42.092938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a7cb9-a42b-4a8a-8102-939d640475cd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:42.099064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:42.099785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:42.100148 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:42.100483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] PATH_INFO: `/v3/projects/251eaac1952247f9ad304278019eb7cf` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:42.101698 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] RBAC: Authorizing `identity:delete_project(project_id=251eaac1952247f9ad304278019eb7cf)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:42.103658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:42.156043 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '251eaac1952247f9ad304278019eb7cf', 'request_id': 'req-d5bc27f5-5c5b-476b-8b6a-29624488f21c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:42.162558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '251eaac1952247f9ad304278019eb7cf', 'request_id': 'req-d5bc27f5-5c5b-476b-8b6a-29624488f21c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:42.168920 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '251eaac1952247f9ad304278019eb7cf', 'request_id': 'req-d5bc27f5-5c5b-476b-8b6a-29624488f21c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:42.184001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] The token cache is being invalidate because project 251eaac1952247f9ad304278019eb7cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:42.184285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:42.184959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:42.186588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 989/1973] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:42 2022] DELETE /identity/v3/projects/251eaac1952247f9ad304278019eb7cf => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:42.190788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee10fa-c638-48a5-810c-75b55544af23 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:42.200692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee10fa-c638-48a5-810c-75b55544af23 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:42.221869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:42.267109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:42.267776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:42.268051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:42.268316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] PATH_INFO: `/v3/users/96dfb70bb97a4c32b68cc26769d94351` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:42.269604 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96dfb70bb97a4c32b68cc26769d94351)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:42.271491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:42.321316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96dfb70bb97a4c32b68cc26769d94351'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:42.321977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96dfb70bb97a4c32b68cc26769d94351'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:42.331756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96dfb70bb97a4c32b68cc26769d94351'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:42.352313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96dfb70bb97a4c32b68cc26769d94351'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:42.353469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96dfb70bb97a4c32b68cc26769d94351'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:42.371127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 985/1974] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:19:42 2022] DELETE /identity/v3/users/96dfb70bb97a4c32b68cc26769d94351 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:19:42.909334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:42.911106 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:42.933293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc27f5-5c5b-476b-8b6a-29624488f21c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:42.959668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:43.009593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:43.010273 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:43.010561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:43.010857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] PATH_INFO: `/v3/projects/8fa5357b890a4bcebc58466125d7ea2e` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:43.011982 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa5357b890a4bcebc58466125d7ea2e)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:43.013986 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:43.071135 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa5357b890a4bcebc58466125d7ea2e', 'request_id': 'req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.078192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa5357b890a4bcebc58466125d7ea2e', 'request_id': 'req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.084187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa5357b890a4bcebc58466125d7ea2e', 'request_id': 'req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.101793 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] The token cache is being invalidate because project 8fa5357b890a4bcebc58466125d7ea2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:43.102132 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.102798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.104433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 990/1975] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:42 2022] DELETE /identity/v3/projects/8fa5357b890a4bcebc58466125d7ea2e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:43.108785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:43.110308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:43.123755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c17449-7b3b-47b8-9db9-2f06d2b0f6b4 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:43.182905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fd486-7409-4f56-8f5d-6dda08171465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:43.183553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fd486-7409-4f56-8f5d-6dda08171465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:43.183829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fd486-7409-4f56-8f5d-6dda08171465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:43.184505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fd486-7409-4f56-8f5d-6dda08171465 admin admin] PATH_INFO: `/v3/users/573322e9d54d48838b90543b69c41527` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:43.185458 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fd486-7409-4f56-8f5d-6dda08171465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=573322e9d54d48838b90543b69c41527)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:43.187499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fd486-7409-4f56-8f5d-6dda08171465 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:43.253822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:43.256687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-950fd486-7409-4f56-8f5d-6dda08171465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '573322e9d54d48838b90543b69c41527'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.257732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-950fd486-7409-4f56-8f5d-6dda08171465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '573322e9d54d48838b90543b69c41527'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.266881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-950fd486-7409-4f56-8f5d-6dda08171465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '573322e9d54d48838b90543b69c41527'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.279546 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-950fd486-7409-4f56-8f5d-6dda08171465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '573322e9d54d48838b90543b69c41527'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.280240 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-950fd486-7409-4f56-8f5d-6dda08171465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '573322e9d54d48838b90543b69c41527'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.295665 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 986/1976] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:19:43 2022] DELETE /identity/v3/users/573322e9d54d48838b90543b69c41527 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:19:43.798579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:43.800779 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:43.815516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee910e70-beb7-4248-a7a6-cb7eb4fa6b61 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:43.873578 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19177aa9-51fd-4d99-9b69-cbadbdda383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:43.874221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19177aa9-51fd-4d99-9b69-cbadbdda383f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:43.874492 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19177aa9-51fd-4d99-9b69-cbadbdda383f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:43.874756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19177aa9-51fd-4d99-9b69-cbadbdda383f admin admin] PATH_INFO: `/v3/projects/22aceb00f5194e7ab93ab70bcb9b9e27` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:43.875876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19177aa9-51fd-4d99-9b69-cbadbdda383f admin admin] RBAC: Authorizing `identity:delete_project(project_id=22aceb00f5194e7ab93ab70bcb9b9e27)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:43.877835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19177aa9-51fd-4d99-9b69-cbadbdda383f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:43.925759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-19177aa9-51fd-4d99-9b69-cbadbdda383f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22aceb00f5194e7ab93ab70bcb9b9e27', 'request_id': 'req-19177aa9-51fd-4d99-9b69-cbadbdda383f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.932177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-19177aa9-51fd-4d99-9b69-cbadbdda383f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22aceb00f5194e7ab93ab70bcb9b9e27', 'request_id': 'req-19177aa9-51fd-4d99-9b69-cbadbdda383f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.933416 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:43.939146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-19177aa9-51fd-4d99-9b69-cbadbdda383f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22aceb00f5194e7ab93ab70bcb9b9e27', 'request_id': 'req-19177aa9-51fd-4d99-9b69-cbadbdda383f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.955899 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-19177aa9-51fd-4d99-9b69-cbadbdda383f admin admin] The token cache is being invalidate because project 22aceb00f5194e7ab93ab70bcb9b9e27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:43.956190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-19177aa9-51fd-4d99-9b69-cbadbdda383f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.956850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-19177aa9-51fd-4d99-9b69-cbadbdda383f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:43.958354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 991/1977] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:43 2022] DELETE /identity/v3/projects/22aceb00f5194e7ab93ab70bcb9b9e27 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:43.964514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab52e1-38f9-46c4-857f-601c8ea3d218 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:43.964828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab52e1-38f9-46c4-857f-601c8ea3d218 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:43.965098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab52e1-38f9-46c4-857f-601c8ea3d218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:43.985746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b3ab52e1-38f9-46c4-857f-601c8ea3d218 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:43.986945 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b3ab52e1-38f9-46c4-857f-601c8ea3d218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:44.047936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 987/1978] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:44.057219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc6be3-7c94-4134-a7c0-0d6b10404132 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.057593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc6be3-7c94-4134-a7c0-0d6b10404132 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.057949 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc6be3-7c94-4134-a7c0-0d6b10404132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.081475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d9dc6be3-7c94-4134-a7c0-0d6b10404132 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:44.083409 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d9dc6be3-7c94-4134-a7c0-0d6b10404132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:44.096338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 992/1979] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:44.100853 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab52e1-38f9-46c4-857f-601c8ea3d218 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:44.111308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab52e1-38f9-46c4-857f-601c8ea3d218 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:44.118220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec381c6e-3518-4a56-b5fe-10876846176c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:44.118835 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec381c6e-3518-4a56-b5fe-10876846176c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.119107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec381c6e-3518-4a56-b5fe-10876846176c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.119369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec381c6e-3518-4a56-b5fe-10876846176c admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.120026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec381c6e-3518-4a56-b5fe-10876846176c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:44.120324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec381c6e-3518-4a56-b5fe-10876846176c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:44.121969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec381c6e-3518-4a56-b5fe-10876846176c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:44.135078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 988/1980] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:19:44.139831 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc6be3-7c94-4134-a7c0-0d6b10404132 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:44.149883 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc6be3-7c94-4134-a7c0-0d6b10404132 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:44.155387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1f76a-4ab0-4f7a-b743-d5ca6c52e7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:44.156012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1f76a-4ab0-4f7a-b743-d5ca6c52e7ce admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.156285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1f76a-4ab0-4f7a-b743-d5ca6c52e7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.156550 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1f76a-4ab0-4f7a-b743-d5ca6c52e7ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.157233 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1f76a-4ab0-4f7a-b743-d5ca6c52e7ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:44.159142 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1f76a-4ab0-4f7a-b743-d5ca6c52e7ce admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:44.189647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 993/1981] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:44 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:44.193930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec381c6e-3518-4a56-b5fe-10876846176c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:44.203715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec381c6e-3518-4a56-b5fe-10876846176c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:44.266751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b32d9d-a5e0-43fa-84ed-f792f5a4d833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:44.267365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b32d9d-a5e0-43fa-84ed-f792f5a4d833 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.267635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b32d9d-a5e0-43fa-84ed-f792f5a4d833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.267897 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b32d9d-a5e0-43fa-84ed-f792f5a4d833 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.268526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b32d9d-a5e0-43fa-84ed-f792f5a4d833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:44.270461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b32d9d-a5e0-43fa-84ed-f792f5a4d833 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:44.294122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 989/1982] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:44 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:44.298681 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1f76a-4ab0-4f7a-b743-d5ca6c52e7ce admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:44.309204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1f76a-4ab0-4f7a-b743-d5ca6c52e7ce admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:44.314195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8b509-ca87-4d34-a1f4-ec7892681ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:44.314804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8b509-ca87-4d34-a1f4-ec7892681ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.315097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8b509-ca87-4d34-a1f4-ec7892681ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.315368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8b509-ca87-4d34-a1f4-ec7892681ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.316133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8b509-ca87-4d34-a1f4-ec7892681ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:44.317762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8b509-ca87-4d34-a1f4-ec7892681ccf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:44.328724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 994/1983] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:44.333023 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b32d9d-a5e0-43fa-84ed-f792f5a4d833 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:44.343686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b32d9d-a5e0-43fa-84ed-f792f5a4d833 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:44.348418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb193bb8-9a58-4b83-829d-480371814387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:44.349220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb193bb8-9a58-4b83-829d-480371814387 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.349491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb193bb8-9a58-4b83-829d-480371814387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.350359 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb193bb8-9a58-4b83-829d-480371814387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.351020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb193bb8-9a58-4b83-829d-480371814387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:44.352625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb193bb8-9a58-4b83-829d-480371814387 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:44.364571 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 990/1984] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:44.369227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8b509-ca87-4d34-a1f4-ec7892681ccf admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:44.379954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8b509-ca87-4d34-a1f4-ec7892681ccf admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:44.384920 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce985e07-e4a6-4dab-9569-61786437f1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:44.389664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce985e07-e4a6-4dab-9569-61786437f1af admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.390036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce985e07-e4a6-4dab-9569-61786437f1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.390384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce985e07-e4a6-4dab-9569-61786437f1af admin admin] PATH_INFO: `/v3/projects/c3a9c7b0f6244b9ba7cb0ae899119cac/users/b74b12d09e304465b51017a801696ec5/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.405654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce985e07-e4a6-4dab-9569-61786437f1af admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a9c7b0f6244b9ba7cb0ae899119cac, user_id=b74b12d09e304465b51017a801696ec5, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:44.408366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce985e07-e4a6-4dab-9569-61786437f1af admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:44.422912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 995/1985] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:44 2022] PUT /identity/v3/projects/c3a9c7b0f6244b9ba7cb0ae899119cac/users/b74b12d09e304465b51017a801696ec5/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:44.434139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30b693-fec2-43fc-8711-a49c9a601223 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.434422 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30b693-fec2-43fc-8711-a49c9a601223 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.434694 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30b693-fec2-43fc-8711-a49c9a601223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.461476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3d30b693-fec2-43fc-8711-a49c9a601223 None None] MFA Rules not processed for user `b74b12d09e304465b51017a801696ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:44.462664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3d30b693-fec2-43fc-8711-a49c9a601223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:44.475403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:44.530800 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 991/1986] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:44.535472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce985e07-e4a6-4dab-9569-61786437f1af admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:44.599175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce985e07-e4a6-4dab-9569-61786437f1af admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:44.604320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a64591-e162-44f3-9b41-5a9dfb3f6824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:44.604957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a64591-e162-44f3-9b41-5a9dfb3f6824 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.605230 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a64591-e162-44f3-9b41-5a9dfb3f6824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.605496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a64591-e162-44f3-9b41-5a9dfb3f6824 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.606131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a64591-e162-44f3-9b41-5a9dfb3f6824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:44.608104 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a64591-e162-44f3-9b41-5a9dfb3f6824 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:44.637361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 996/1987] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:44 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:44.641088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30b693-fec2-43fc-8711-a49c9a601223 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:44.651956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30b693-fec2-43fc-8711-a49c9a601223 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:44.710369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc426c8-9839-4fe0-8d03-9a3610b3c966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:44.710971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc426c8-9839-4fe0-8d03-9a3610b3c966 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.711240 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc426c8-9839-4fe0-8d03-9a3610b3c966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.711505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc426c8-9839-4fe0-8d03-9a3610b3c966 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.712117 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc426c8-9839-4fe0-8d03-9a3610b3c966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:44.714040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc426c8-9839-4fe0-8d03-9a3610b3c966 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:44.738430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 992/1988] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:44 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:44.742699 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a64591-e162-44f3-9b41-5a9dfb3f6824 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:44.753166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a64591-e162-44f3-9b41-5a9dfb3f6824 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:44.758182 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0bbe2-0431-4e18-8e44-0835f7d1c249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:44.758777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0bbe2-0431-4e18-8e44-0835f7d1c249 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.759041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0bbe2-0431-4e18-8e44-0835f7d1c249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.759277 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0bbe2-0431-4e18-8e44-0835f7d1c249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.759941 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0bbe2-0431-4e18-8e44-0835f7d1c249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:44.761579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0bbe2-0431-4e18-8e44-0835f7d1c249 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:44.772184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 997/1989] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:44.776501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc426c8-9839-4fe0-8d03-9a3610b3c966 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:44.788433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc426c8-9839-4fe0-8d03-9a3610b3c966 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:44.794211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca846606-1920-4a62-8eca-97776600e3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:44.794893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca846606-1920-4a62-8eca-97776600e3a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.795188 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca846606-1920-4a62-8eca-97776600e3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.795438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca846606-1920-4a62-8eca-97776600e3a5 admin admin] PATH_INFO: `/v3/projects/63e679306f1f46979fee8a72671135a0/users/c2add5d07eec40e79bb190b44d7e538e/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.811874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca846606-1920-4a62-8eca-97776600e3a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e679306f1f46979fee8a72671135a0, user_id=c2add5d07eec40e79bb190b44d7e538e, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:44.814575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca846606-1920-4a62-8eca-97776600e3a5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:44.829497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 993/1990] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:44 2022] PUT /identity/v3/projects/63e679306f1f46979fee8a72671135a0/users/c2add5d07eec40e79bb190b44d7e538e/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:44.833687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0bbe2-0431-4e18-8e44-0835f7d1c249 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:44.844116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0bbe2-0431-4e18-8e44-0835f7d1c249 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:44.893366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:44.903911 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ba1a1-e14e-4ceb-ae6b-25c3b1a5856d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:44.904976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ba1a1-e14e-4ceb-ae6b-25c3b1a5856d admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.905329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ba1a1-e14e-4ceb-ae6b-25c3b1a5856d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.905874 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ba1a1-e14e-4ceb-ae6b-25c3b1a5856d admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.906540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ba1a1-e14e-4ceb-ae6b-25c3b1a5856d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:44.908180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ba1a1-e14e-4ceb-ae6b-25c3b1a5856d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:44.918394 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 998/1991] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:44.922897 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca846606-1920-4a62-8eca-97776600e3a5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:44.933396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca846606-1920-4a62-8eca-97776600e3a5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:44.938753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5cb25-8131-43d8-89ff-25a9d6470944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:44.939386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5cb25-8131-43d8-89ff-25a9d6470944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.939684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5cb25-8131-43d8-89ff-25a9d6470944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.939956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e5cb25-8131-43d8-89ff-25a9d6470944 admin admin] PATH_INFO: `/v3/projects/63e679306f1f46979fee8a72671135a0/users/c2add5d07eec40e79bb190b44d7e538e/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.942094 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5cb25-8131-43d8-89ff-25a9d6470944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e679306f1f46979fee8a72671135a0, user_id=c2add5d07eec40e79bb190b44d7e538e, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:44.944775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e5cb25-8131-43d8-89ff-25a9d6470944 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:44.955918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 994/1992] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:44 2022] PUT /identity/v3/projects/63e679306f1f46979fee8a72671135a0/users/c2add5d07eec40e79bb190b44d7e538e/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:19:44.967317 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461f63-399c-4283-9c30-3d395d113599 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:44.967871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461f63-399c-4283-9c30-3d395d113599 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:44.967871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4461f63-399c-4283-9c30-3d395d113599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:44.996883 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c4461f63-399c-4283-9c30-3d395d113599 None None] MFA Rules not processed for user `c2add5d07eec40e79bb190b44d7e538e`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:44.998168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c4461f63-399c-4283-9c30-3d395d113599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:45.072930 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 999/1993] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:45.084459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5cb25-8131-43d8-89ff-25a9d6470944 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:45.086709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c0e5cb25-8131-43d8-89ff-25a9d6470944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:45.154871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e5cb25-8131-43d8-89ff-25a9d6470944 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:45.161069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b224453-9809-40b9-b8b0-f1f60b675f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:45.161951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b224453-9809-40b9-b8b0-f1f60b675f4e service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:45.162337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b224453-9809-40b9-b8b0-f1f60b675f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:45.162647 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b224453-9809-40b9-b8b0-f1f60b675f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:45.176470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b224453-9809-40b9-b8b0-f1f60b675f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:45.178000 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b224453-9809-40b9-b8b0-f1f60b675f4e service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:45.188518 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 995/1994] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:45.298990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4461f63-399c-4283-9c30-3d395d113599 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:45.308895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4461f63-399c-4283-9c30-3d395d113599 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:45.312908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0aec6-dd23-4b77-a8a5-f508e2e57d34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:45.313511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0aec6-dd23-4b77-a8a5-f508e2e57d34 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:45.313781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0aec6-dd23-4b77-a8a5-f508e2e57d34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:45.314047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0aec6-dd23-4b77-a8a5-f508e2e57d34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:45.379412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0aec6-dd23-4b77-a8a5-f508e2e57d34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:45.380884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af0aec6-dd23-4b77-a8a5-f508e2e57d34 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:45.391676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1000/1995] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:52.286354 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b224453-9809-40b9-b8b0-f1f60b675f4e service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:52.303267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:52.358724 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b224453-9809-40b9-b8b0-f1f60b675f4e service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:52.364977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:52.365790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:52.366095 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:52.366368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] PATH_INFO: `/v3/users/b74b12d09e304465b51017a801696ec5` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:52.367754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b74b12d09e304465b51017a801696ec5)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:52.369745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:52.419107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b74b12d09e304465b51017a801696ec5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:52.419739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b74b12d09e304465b51017a801696ec5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:52.426623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b74b12d09e304465b51017a801696ec5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:52.438061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b74b12d09e304465b51017a801696ec5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:52.438874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b74b12d09e304465b51017a801696ec5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:52.454264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 996/1996] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:19:52 2022] DELETE /identity/v3/users/b74b12d09e304465b51017a801696ec5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:52.459019 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424a4e0-5cca-450c-88a5-72e404d7c187 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:52.459529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424a4e0-5cca-450c-88a5-72e404d7c187 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:52.459529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424a4e0-5cca-450c-88a5-72e404d7c187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:52.479716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6424a4e0-5cca-450c-88a5-72e404d7c187 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:52.480894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6424a4e0-5cca-450c-88a5-72e404d7c187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:52.573817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1001/1997] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:19:52.583642 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:52.585471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:52.655873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba901b-f219-49d5-addb-677b72d6a67d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:52.659883 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c983a45-8c9a-4eda-9770-113e82bfac8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:52.660512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c983a45-8c9a-4eda-9770-113e82bfac8d service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:52.660857 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c983a45-8c9a-4eda-9770-113e82bfac8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:52.661089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c983a45-8c9a-4eda-9770-113e82bfac8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:52.671716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c983a45-8c9a-4eda-9770-113e82bfac8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:52.677452 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c983a45-8c9a-4eda-9770-113e82bfac8d service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:52.688408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 997/1998] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:52.741271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424a4e0-5cca-450c-88a5-72e404d7c187 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:52.742821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6424a4e0-5cca-450c-88a5-72e404d7c187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:52.756614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424a4e0-5cca-450c-88a5-72e404d7c187 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:52.763204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:52.764066 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:52.764300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:52.764676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] PATH_INFO: `/v3/users/5d641b3307e64a11b6cd4f18f02248d2` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:52.765933 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d641b3307e64a11b6cd4f18f02248d2)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:52.768003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:52.774381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:52.830713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d641b3307e64a11b6cd4f18f02248d2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:52.858482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d641b3307e64a11b6cd4f18f02248d2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:52.884975 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d641b3307e64a11b6cd4f18f02248d2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:52.900983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d641b3307e64a11b6cd4f18f02248d2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:52.901910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d641b3307e64a11b6cd4f18f02248d2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:52.914404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1002/1999] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:19:52 2022] DELETE /identity/v3/users/5d641b3307e64a11b6cd4f18f02248d2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:52.916049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fee704-f68e-47d5-9322-56eab79f1f07 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:52.916334 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fee704-f68e-47d5-9322-56eab79f1f07 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:52.916628 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fee704-f68e-47d5-9322-56eab79f1f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:52.950595 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-08fee704-f68e-47d5-9322-56eab79f1f07 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:52.951935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-08fee704-f68e-47d5-9322-56eab79f1f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:52.985389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:53.031400 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 998/2000] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:53.041942 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:53.043524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:53.118398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f8c15-403f-472a-aa45-0aa968a41010 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:53.130079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8f921-23cf-4ab4-8811-d1b84aa5973b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:53.130820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8f921-23cf-4ab4-8811-d1b84aa5973b service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:53.131302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8f921-23cf-4ab4-8811-d1b84aa5973b service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:53.131590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8f921-23cf-4ab4-8811-d1b84aa5973b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:53.141850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8f921-23cf-4ab4-8811-d1b84aa5973b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:53.143302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8f921-23cf-4ab4-8811-d1b84aa5973b service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:53.145684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fee704-f68e-47d5-9322-56eab79f1f07 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:53.147200 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-08fee704-f68e-47d5-9322-56eab79f1f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:53.167695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1003/2001] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:53.174486 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fee704-f68e-47d5-9322-56eab79f1f07 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:53.180074 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:53.181292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:53.181292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:53.181504 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] PATH_INFO: `/v3/projects/c3a9c7b0f6244b9ba7cb0ae899119cac` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:53.182734 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a9c7b0f6244b9ba7cb0ae899119cac)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:53.184822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:53.267268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a9c7b0f6244b9ba7cb0ae899119cac', 'request_id': 'req-1cf10d87-ce52-4a5b-8e05-e88118888ad7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.278591 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a9c7b0f6244b9ba7cb0ae899119cac', 'request_id': 'req-1cf10d87-ce52-4a5b-8e05-e88118888ad7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.286534 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a9c7b0f6244b9ba7cb0ae899119cac', 'request_id': 'req-1cf10d87-ce52-4a5b-8e05-e88118888ad7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.298564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] The token cache is being invalidate because project c3a9c7b0f6244b9ba7cb0ae899119cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:53.299996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.300736 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.301739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8f921-23cf-4ab4-8811-d1b84aa5973b service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:53.302911 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 999/2002] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:53 2022] DELETE /identity/v3/projects/c3a9c7b0f6244b9ba7cb0ae899119cac => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:19:53.303841 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b6c8f921-23cf-4ab4-8811-d1b84aa5973b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:53.306886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:53.309061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:53.325084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8f921-23cf-4ab4-8811-d1b84aa5973b service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:53.389026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf10d87-ce52-4a5b-8e05-e88118888ad7 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:53.394732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:53.395417 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:53.395661 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:53.395927 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] PATH_INFO: `/v3/users/c2add5d07eec40e79bb190b44d7e538e` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:53.397163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2add5d07eec40e79bb190b44d7e538e)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:53.399125 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:53.415173 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d6156-ace0-44a4-9b26-af54e79d6075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:53.416051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d6156-ace0-44a4-9b26-af54e79d6075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:53.416330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d6156-ace0-44a4-9b26-af54e79d6075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:53.416618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d6156-ace0-44a4-9b26-af54e79d6075 admin admin] PATH_INFO: `/v3/users/7937f0c95b1241e9a7bb4914f4f7d391` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:53.418713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d6156-ace0-44a4-9b26-af54e79d6075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7937f0c95b1241e9a7bb4914f4f7d391)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:53.420661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d6156-ace0-44a4-9b26-af54e79d6075 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:53.462443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2add5d07eec40e79bb190b44d7e538e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.463442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2add5d07eec40e79bb190b44d7e538e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.471869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2add5d07eec40e79bb190b44d7e538e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.473902 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:53.490215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:53.490781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2add5d07eec40e79bb190b44d7e538e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.491433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2add5d07eec40e79bb190b44d7e538e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.505715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1000/2003] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:19:53 2022] DELETE /identity/v3/users/c2add5d07eec40e79bb190b44d7e538e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:53.511801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5b0d6156-ace0-44a4-9b26-af54e79d6075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7937f0c95b1241e9a7bb4914f4f7d391'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.514652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:53.516090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:53.533749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5b0d6156-ace0-44a4-9b26-af54e79d6075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7937f0c95b1241e9a7bb4914f4f7d391'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.569421 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5b0d6156-ace0-44a4-9b26-af54e79d6075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7937f0c95b1241e9a7bb4914f4f7d391'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.580543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5b0d6156-ace0-44a4-9b26-af54e79d6075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7937f0c95b1241e9a7bb4914f4f7d391'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.581924 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5b0d6156-ace0-44a4-9b26-af54e79d6075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7937f0c95b1241e9a7bb4914f4f7d391'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.592125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1004/2004] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:19:53 2022] DELETE /identity/v3/users/7937f0c95b1241e9a7bb4914f4f7d391 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:19:53.597657 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a428ba0-6f74-49bc-b696-b624fe06f16b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:53.598006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a428ba0-6f74-49bc-b696-b624fe06f16b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:53.598602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a428ba0-6f74-49bc-b696-b624fe06f16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:53.624948 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988ae47-3e82-4658-b301-293c7904b2b3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:53.632818 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7a428ba0-6f74-49bc-b696-b624fe06f16b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:53.634172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7a428ba0-6f74-49bc-b696-b624fe06f16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:53.705324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e469616-def6-49de-adad-813736ed2673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:53.706045 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e469616-def6-49de-adad-813736ed2673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:53.706304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e469616-def6-49de-adad-813736ed2673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:53.706624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e469616-def6-49de-adad-813736ed2673 admin admin] PATH_INFO: `/v3/projects/e161ec22343e45a8b0ace2c6611d1bf1` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:53.710512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1005/2005] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:19:53.710672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e469616-def6-49de-adad-813736ed2673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e161ec22343e45a8b0ace2c6611d1bf1)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:53.710672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e469616-def6-49de-adad-813736ed2673 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:53.722228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a428ba0-6f74-49bc-b696-b624fe06f16b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:53.724162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7a428ba0-6f74-49bc-b696-b624fe06f16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:53.791008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2e469616-def6-49de-adad-813736ed2673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e161ec22343e45a8b0ace2c6611d1bf1', 'request_id': 'req-2e469616-def6-49de-adad-813736ed2673'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.799857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a428ba0-6f74-49bc-b696-b624fe06f16b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:53.805275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2e469616-def6-49de-adad-813736ed2673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e161ec22343e45a8b0ace2c6611d1bf1', 'request_id': 'req-2e469616-def6-49de-adad-813736ed2673'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.815551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2e469616-def6-49de-adad-813736ed2673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e161ec22343e45a8b0ace2c6611d1bf1', 'request_id': 'req-2e469616-def6-49de-adad-813736ed2673'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.829917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2e469616-def6-49de-adad-813736ed2673 admin admin] The token cache is being invalidate because project e161ec22343e45a8b0ace2c6611d1bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:53.830872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2e469616-def6-49de-adad-813736ed2673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.832192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2e469616-def6-49de-adad-813736ed2673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:53.836875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1001/2006] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:53 2022] DELETE /identity/v3/projects/e161ec22343e45a8b0ace2c6611d1bf1 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:53.842795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb3814-2812-405c-82f0-60fe7efe1bb0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:53.843130 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb3814-2812-405c-82f0-60fe7efe1bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:53.844047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb3814-2812-405c-82f0-60fe7efe1bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:53.874810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45d759-728e-4a6a-8ba3-7fc391b00001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:53.875889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45d759-728e-4a6a-8ba3-7fc391b00001 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:53.876474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45d759-728e-4a6a-8ba3-7fc391b00001 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:53.877096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45d759-728e-4a6a-8ba3-7fc391b00001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:53.877495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-eabb3814-2812-405c-82f0-60fe7efe1bb0 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:53.878898 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-eabb3814-2812-405c-82f0-60fe7efe1bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:53.881626 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4a45d759-728e-4a6a-8ba3-7fc391b00001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:53.895024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:53.907147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:53.959888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45d759-728e-4a6a-8ba3-7fc391b00001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:53.961388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45d759-728e-4a6a-8ba3-7fc391b00001 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:53.970816 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1002/2007] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:53.977106 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb3814-2812-405c-82f0-60fe7efe1bb0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:53.977218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1006/2008] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:19:53.977308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-eabb3814-2812-405c-82f0-60fe7efe1bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:53.978447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56819649-19fe-4a5b-b89e-245ee67468a3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:53.978723 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56819649-19fe-4a5b-b89e-245ee67468a3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:53.978988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56819649-19fe-4a5b-b89e-245ee67468a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:53.985234 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb3814-2812-405c-82f0-60fe7efe1bb0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:53.990461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:53.991249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:53.991387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:53.991636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71 admin admin] PATH_INFO: `/v3/projects/63e679306f1f46979fee8a72671135a0` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:53.992986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e679306f1f46979fee8a72671135a0)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:53.995762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.004818 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-56819649-19fe-4a5b-b89e-245ee67468a3 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:54.006031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-56819649-19fe-4a5b-b89e-245ee67468a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:54.016625 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1007/2009] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:19:54.021607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56819649-19fe-4a5b-b89e-245ee67468a3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.031043 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56819649-19fe-4a5b-b89e-245ee67468a3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.035958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f5016-9edd-4e19-b7a1-8c7886b7a368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.036536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f5016-9edd-4e19-b7a1-8c7886b7a368 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.036853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f5016-9edd-4e19-b7a1-8c7886b7a368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.037125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f5016-9edd-4e19-b7a1-8c7886b7a368 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.038018 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f5016-9edd-4e19-b7a1-8c7886b7a368 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:54.038295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f5016-9edd-4e19-b7a1-8c7886b7a368 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.039866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f5016-9edd-4e19-b7a1-8c7886b7a368 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.054242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1008/2010] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:19:54.058810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f5016-9edd-4e19-b7a1-8c7886b7a368 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.069290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e679306f1f46979fee8a72671135a0', 'request_id': 'req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.069532 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f5016-9edd-4e19-b7a1-8c7886b7a368 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.081158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e679306f1f46979fee8a72671135a0', 'request_id': 'req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.090119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e679306f1f46979fee8a72671135a0', 'request_id': 'req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.106566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71 admin admin] The token cache is being invalidate because project 63e679306f1f46979fee8a72671135a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:54.106863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.107554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cafb3ea4-f684-4e80-a5d3-7d17f3eceb71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.109204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1003/2011] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:53 2022] DELETE /identity/v3/projects/63e679306f1f46979fee8a72671135a0 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:54.116565 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b91df-80a4-4885-acfe-43576ce12b98 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.120938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b91df-80a4-4885-acfe-43576ce12b98 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.121482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b91df-80a4-4885-acfe-43576ce12b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.157249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c34b91df-80a4-4885-acfe-43576ce12b98 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:54.158566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c34b91df-80a4-4885-acfe-43576ce12b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:54.159095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b56255-e7f7-4605-b9cf-eae9da7c94ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.160092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b56255-e7f7-4605-b9cf-eae9da7c94ce admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.160748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b56255-e7f7-4605-b9cf-eae9da7c94ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.161498 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b56255-e7f7-4605-b9cf-eae9da7c94ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.162769 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b56255-e7f7-4605-b9cf-eae9da7c94ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.164997 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b56255-e7f7-4605-b9cf-eae9da7c94ce admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.175848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1004/2012] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:54.184397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55bb2d-acf1-4633-9ede-6bb00c2dee63 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.184970 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55bb2d-acf1-4633-9ede-6bb00c2dee63 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.186551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55bb2d-acf1-4633-9ede-6bb00c2dee63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.212457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1009/2013] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:54 2022] POST /identity/v3/projects => generated 369 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:19:54.213505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b56255-e7f7-4605-b9cf-eae9da7c94ce admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.214980 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-9c55bb2d-acf1-4633-9ede-6bb00c2dee63 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:54.215346 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f4b56255-e7f7-4605-b9cf-eae9da7c94ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:54.216146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9c55bb2d-acf1-4633-9ede-6bb00c2dee63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:54.286674 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1005/2014] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:54.290811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55bb2d-acf1-4633-9ede-6bb00c2dee63 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.301884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55bb2d-acf1-4633-9ede-6bb00c2dee63 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.307818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0dc16-e72d-4a91-9807-a582422d2925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.309143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0dc16-e72d-4a91-9807-a582422d2925 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.309597 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0dc16-e72d-4a91-9807-a582422d2925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.310038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd0dc16-e72d-4a91-9807-a582422d2925 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.310873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0dc16-e72d-4a91-9807-a582422d2925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:54.311388 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0dc16-e72d-4a91-9807-a582422d2925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.313071 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd0dc16-e72d-4a91-9807-a582422d2925 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.317338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b56255-e7f7-4605-b9cf-eae9da7c94ce admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.325606 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee484ab-5d0b-4a13-9e1d-43c567ffb802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.326239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee484ab-5d0b-4a13-9e1d-43c567ffb802 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.326510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee484ab-5d0b-4a13-9e1d-43c567ffb802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.326775 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee484ab-5d0b-4a13-9e1d-43c567ffb802 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.327418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee484ab-5d0b-4a13-9e1d-43c567ffb802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.329499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee484ab-5d0b-4a13-9e1d-43c567ffb802 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.331850 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1006/2015] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:19:54.336702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0dc16-e72d-4a91-9807-a582422d2925 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.354015 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd0dc16-e72d-4a91-9807-a582422d2925 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.362240 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafae60-a39f-408e-ab9a-4afcb58d481a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.362901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafae60-a39f-408e-ab9a-4afcb58d481a admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.364218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafae60-a39f-408e-ab9a-4afcb58d481a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.364487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafae60-a39f-408e-ab9a-4afcb58d481a admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.367736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1010/2016] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:54 2022] POST /identity/v3/users => generated 285 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:54.367890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafae60-a39f-408e-ab9a-4afcb58d481a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.367890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafae60-a39f-408e-ab9a-4afcb58d481a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.368887 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee484ab-5d0b-4a13-9e1d-43c567ffb802 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.381684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee484ab-5d0b-4a13-9e1d-43c567ffb802 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.387452 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8483aa-e5cf-4d1b-829c-9c6a944e4f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.388068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8483aa-e5cf-4d1b-829c-9c6a944e4f34 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.388335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8483aa-e5cf-4d1b-829c-9c6a944e4f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.388621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8483aa-e5cf-4d1b-829c-9c6a944e4f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.389400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8483aa-e5cf-4d1b-829c-9c6a944e4f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.389518 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:54.390993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8483aa-e5cf-4d1b-829c-9c6a944e4f34 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.394816 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:54.406108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1011/2017] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:54.407771 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8483aa-e5cf-4d1b-829c-9c6a944e4f34 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.411924 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4d8483aa-e5cf-4d1b-829c-9c6a944e4f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:54.414103 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1007/2018] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:54 2022] POST /identity/v3/projects => generated 377 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:54.415688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafae60-a39f-408e-ab9a-4afcb58d481a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.430116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8483aa-e5cf-4d1b-829c-9c6a944e4f34 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.481415 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafae60-a39f-408e-ab9a-4afcb58d481a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.486064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ed654-8ff9-4df3-8ad0-3336594e84e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.486310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.486677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ed654-8ff9-4df3-8ad0-3336594e84e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.486966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.487094 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ed654-8ff9-4df3-8ad0-3336594e84e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.487255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ed654-8ff9-4df3-8ad0-3336594e84e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.487349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.487508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] PATH_INFO: `/v3/projects/c89c1fb05f6f4662ac6b0cf26673ed13` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.487869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ed654-8ff9-4df3-8ad0-3336594e84e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.489001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c89c1fb05f6f4662ac6b0cf26673ed13)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.489848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ed654-8ff9-4df3-8ad0-3336594e84e2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.490911 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.500575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1008/2019] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:54.501409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ed654-8ff9-4df3-8ad0-3336594e84e2 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.511329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ed654-8ff9-4df3-8ad0-3336594e84e2 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.516711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0777e-92fb-4bcb-96d9-4fe6048b7f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.517291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0777e-92fb-4bcb-96d9-4fe6048b7f8f admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.517562 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0777e-92fb-4bcb-96d9-4fe6048b7f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.517828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0777e-92fb-4bcb-96d9-4fe6048b7f8f admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.518468 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0777e-92fb-4bcb-96d9-4fe6048b7f8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.520462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0777e-92fb-4bcb-96d9-4fe6048b7f8f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.548727 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1009/2020] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:54 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:54.549241 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0777e-92fb-4bcb-96d9-4fe6048b7f8f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.553277 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c89c1fb05f6f4662ac6b0cf26673ed13', 'request_id': 'req-f7875235-c1f8-450b-bdb7-03fa943a0924'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.559659 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0777e-92fb-4bcb-96d9-4fe6048b7f8f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.563783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c89c1fb05f6f4662ac6b0cf26673ed13', 'request_id': 'req-f7875235-c1f8-450b-bdb7-03fa943a0924'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.569421 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c89c1fb05f6f4662ac6b0cf26673ed13', 'request_id': 'req-f7875235-c1f8-450b-bdb7-03fa943a0924'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.585303 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] The token cache is being invalidate because project c89c1fb05f6f4662ac6b0cf26673ed13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:54.585681 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.586351 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.587926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1012/2021] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:54 2022] DELETE /identity/v3/projects/c89c1fb05f6f4662ac6b0cf26673ed13 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:54.588721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.590055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:54.620806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af424101-bb24-46c6-9915-f3847e4bbdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.621468 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af424101-bb24-46c6-9915-f3847e4bbdd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.621740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af424101-bb24-46c6-9915-f3847e4bbdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.622005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af424101-bb24-46c6-9915-f3847e4bbdd3 admin admin] PATH_INFO: `/v3/projects/efa8ccdc30dc4651b730950a0a5fe8d4/users/159528b6ad49481f86bfbbac6187d584/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.637038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af424101-bb24-46c6-9915-f3847e4bbdd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efa8ccdc30dc4651b730950a0a5fe8d4, user_id=159528b6ad49481f86bfbbac6187d584, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.639936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af424101-bb24-46c6-9915-f3847e4bbdd3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.656616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1010/2022] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:54 2022] PUT /identity/v3/projects/efa8ccdc30dc4651b730950a0a5fe8d4/users/159528b6ad49481f86bfbbac6187d584/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:54.657664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875235-c1f8-450b-bdb7-03fa943a0924 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.657785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af424101-bb24-46c6-9915-f3847e4bbdd3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.659414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-af424101-bb24-46c6-9915-f3847e4bbdd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:54.692508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:54.718343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68f8fe-c100-474a-8dbb-ce8ae3257f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.718964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68f8fe-c100-474a-8dbb-ce8ae3257f05 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.719256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68f8fe-c100-474a-8dbb-ce8ae3257f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.719718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68f8fe-c100-474a-8dbb-ce8ae3257f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.720525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68f8fe-c100-474a-8dbb-ce8ae3257f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.722318 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:54.722456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68f8fe-c100-474a-8dbb-ce8ae3257f05 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.726446 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af424101-bb24-46c6-9915-f3847e4bbdd3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.735875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.736119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1013/2023] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:54.736280 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.736280 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.736280 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] PATH_INFO: `/v3/users/f7c40a8283844922949f1299cc5b95f9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.736522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e0079-5626-4cd6-924d-d02198193a15 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.736522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e0079-5626-4cd6-924d-d02198193a15 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.736522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e0079-5626-4cd6-924d-d02198193a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.737722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7c40a8283844922949f1299cc5b95f9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.740189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.755772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9e2e0079-5626-4cd6-924d-d02198193a15 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:54.757033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9e2e0079-5626-4cd6-924d-d02198193a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:54.764683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1014/2024] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:54.765462 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e0079-5626-4cd6-924d-d02198193a15 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.775050 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e0079-5626-4cd6-924d-d02198193a15 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.779750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62268aaf-09be-469f-9b7a-6b64e075acd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.780369 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62268aaf-09be-469f-9b7a-6b64e075acd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.780657 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62268aaf-09be-469f-9b7a-6b64e075acd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.780913 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62268aaf-09be-469f-9b7a-6b64e075acd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.781590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62268aaf-09be-469f-9b7a-6b64e075acd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.783537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62268aaf-09be-469f-9b7a-6b64e075acd6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.794232 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1015/2025] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:54.795133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62268aaf-09be-469f-9b7a-6b64e075acd6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.797131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-62268aaf-09be-469f-9b7a-6b64e075acd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:54.800718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7c40a8283844922949f1299cc5b95f9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.801333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7c40a8283844922949f1299cc5b95f9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.808154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7c40a8283844922949f1299cc5b95f9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.819951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7c40a8283844922949f1299cc5b95f9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.820669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7c40a8283844922949f1299cc5b95f9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:54.835036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1011/2026] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:19:54 2022] DELETE /identity/v3/users/f7c40a8283844922949f1299cc5b95f9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:54.837412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:54.839406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:54.855533 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62268aaf-09be-469f-9b7a-6b64e075acd6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.910149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f693c-4541-429d-ac7e-7770f79380cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.910759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f693c-4541-429d-ac7e-7770f79380cd service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.911027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f693c-4541-429d-ac7e-7770f79380cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.911388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f693c-4541-429d-ac7e-7770f79380cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.912660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988be54-5753-4cc2-90ef-240ac7379722 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:54.913255 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5e2f693c-4541-429d-ac7e-7770f79380cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:54.917838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3861814-dd52-4b17-a2b5-429ce2a0674b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:54.918513 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3861814-dd52-4b17-a2b5-429ce2a0674b admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.918792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3861814-dd52-4b17-a2b5-429ce2a0674b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.919063 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3861814-dd52-4b17-a2b5-429ce2a0674b admin admin] PATH_INFO: `/v3/projects/189822c3b45a42da97b87c16b0d2e529/users/8ccf6b70a2b845b7ac2547565f04bf56/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:54.926566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f693c-4541-429d-ac7e-7770f79380cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.927994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f693c-4541-429d-ac7e-7770f79380cd service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.937098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3861814-dd52-4b17-a2b5-429ce2a0674b admin admin] RBAC: Authorizing `identity:create_grant(project_id=189822c3b45a42da97b87c16b0d2e529, user_id=8ccf6b70a2b845b7ac2547565f04bf56, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:54.938515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1016/2027] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:54.939923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3861814-dd52-4b17-a2b5-429ce2a0674b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:54.955686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1012/2028] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:54 2022] PUT /identity/v3/projects/189822c3b45a42da97b87c16b0d2e529/users/8ccf6b70a2b845b7ac2547565f04bf56/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:54.967289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ffbc4-8153-4c9d-847e-6edbe04894a9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:54.967653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ffbc4-8153-4c9d-847e-6edbe04894a9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:54.967985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9ffbc4-8153-4c9d-847e-6edbe04894a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.007766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0b9ffbc4-8153-4c9d-847e-6edbe04894a9 None None] MFA Rules not processed for user `8ccf6b70a2b845b7ac2547565f04bf56`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.009196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0b9ffbc4-8153-4c9d-847e-6edbe04894a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.099337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1017/2029] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.104370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3861814-dd52-4b17-a2b5-429ce2a0674b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.160179 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ffbc4-8153-4c9d-847e-6edbe04894a9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.161907 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0b9ffbc4-8153-4c9d-847e-6edbe04894a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.168051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:55.225758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3861814-dd52-4b17-a2b5-429ce2a0674b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.233256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969a9bb-8442-470d-947a-e065ed748f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.235891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969a9bb-8442-470d-947a-e065ed748f8b admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.236489 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969a9bb-8442-470d-947a-e065ed748f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.236790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969a9bb-8442-470d-947a-e065ed748f8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.237553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969a9bb-8442-470d-947a-e065ed748f8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.240776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969a9bb-8442-470d-947a-e065ed748f8b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.248103 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:55.285635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9ffbc4-8153-4c9d-847e-6edbe04894a9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.288268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1013/2030] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/projects => generated 375 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:55.291964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969a9bb-8442-470d-947a-e065ed748f8b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.401676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a68906-373b-4125-ac92-2bd0a0093f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.402357 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a68906-373b-4125-ac92-2bd0a0093f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.402663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a68906-373b-4125-ac92-2bd0a0093f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.402989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a68906-373b-4125-ac92-2bd0a0093f17 admin admin] PATH_INFO: `/v3/projects/a69fbd84f7b24d7581c987a78e471107` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.404914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a68906-373b-4125-ac92-2bd0a0093f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a69fbd84f7b24d7581c987a78e471107)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.408086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a68906-373b-4125-ac92-2bd0a0093f17 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.426588 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969a9bb-8442-470d-947a-e065ed748f8b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.431621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b22c0b-5e45-4811-936e-6a8c60907990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.432329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b22c0b-5e45-4811-936e-6a8c60907990 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.433108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b22c0b-5e45-4811-936e-6a8c60907990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.433394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b22c0b-5e45-4811-936e-6a8c60907990 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.434047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b22c0b-5e45-4811-936e-6a8c60907990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.436010 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b22c0b-5e45-4811-936e-6a8c60907990 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.469361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1014/2031] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:19:55.469361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b22c0b-5e45-4811-936e-6a8c60907990 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.480367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b22c0b-5e45-4811-936e-6a8c60907990 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.493396 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-51a68906-373b-4125-ac92-2bd0a0093f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a69fbd84f7b24d7581c987a78e471107', 'request_id': 'req-51a68906-373b-4125-ac92-2bd0a0093f17'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:55.501860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-51a68906-373b-4125-ac92-2bd0a0093f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a69fbd84f7b24d7581c987a78e471107', 'request_id': 'req-51a68906-373b-4125-ac92-2bd0a0093f17'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:55.508556 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-51a68906-373b-4125-ac92-2bd0a0093f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a69fbd84f7b24d7581c987a78e471107', 'request_id': 'req-51a68906-373b-4125-ac92-2bd0a0093f17'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:55.537896 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-51a68906-373b-4125-ac92-2bd0a0093f17 admin admin] The token cache is being invalidate because project a69fbd84f7b24d7581c987a78e471107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:19:55.538270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-51a68906-373b-4125-ac92-2bd0a0093f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:55.540247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-51a68906-373b-4125-ac92-2bd0a0093f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:55.544688 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1018/2032] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:55 2022] DELETE /identity/v3/projects/a69fbd84f7b24d7581c987a78e471107 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:55.548381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa7dd0-c83b-492f-8cc7-908270153d1b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.549522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa7dd0-c83b-492f-8cc7-908270153d1b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.550830 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa7dd0-c83b-492f-8cc7-908270153d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.574506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fb3dd-0116-43a4-aa12-a5319eca1a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.575152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fb3dd-0116-43a4-aa12-a5319eca1a48 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.575748 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fb3dd-0116-43a4-aa12-a5319eca1a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.576033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fb3dd-0116-43a4-aa12-a5319eca1a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.576983 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fb3dd-0116-43a4-aa12-a5319eca1a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.579327 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fb3dd-0116-43a4-aa12-a5319eca1a48 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.582408 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f5aa7dd0-c83b-492f-8cc7-908270153d1b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.586276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f5aa7dd0-c83b-492f-8cc7-908270153d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.595512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1015/2033] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:55.596382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fb3dd-0116-43a4-aa12-a5319eca1a48 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.598496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b86fb3dd-0116-43a4-aa12-a5319eca1a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.606035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1019/2034] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:19:55.609798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a3a45-0235-4349-80f0-9b4cf93fbae1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.611851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a3a45-0235-4349-80f0-9b4cf93fbae1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.612302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a3a45-0235-4349-80f0-9b4cf93fbae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.613385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fb3dd-0116-43a4-aa12-a5319eca1a48 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.619325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bae275-6e33-4e76-bc2f-f48bbe350186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.620046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bae275-6e33-4e76-bc2f-f48bbe350186 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.620324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bae275-6e33-4e76-bc2f-f48bbe350186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.620620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bae275-6e33-4e76-bc2f-f48bbe350186 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.621270 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bae275-6e33-4e76-bc2f-f48bbe350186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.622980 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bae275-6e33-4e76-bc2f-f48bbe350186 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.641170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1016/2035] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:55.647312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bae275-6e33-4e76-bc2f-f48bbe350186 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.649698 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cc5a3a45-0235-4349-80f0-9b4cf93fbae1 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.650920 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cc5a3a45-0235-4349-80f0-9b4cf93fbae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.668626 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bae275-6e33-4e76-bc2f-f48bbe350186 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.673795 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1020/2036] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:55.674793 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a3be7-dec0-4ca1-8dc9-b7bdddbdb755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.675549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a3be7-dec0-4ca1-8dc9-b7bdddbdb755 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.675910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a3be7-dec0-4ca1-8dc9-b7bdddbdb755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.676283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a3be7-dec0-4ca1-8dc9-b7bdddbdb755 admin admin] PATH_INFO: `/v3/projects/6bbc4217ea2148108dc2165708dc9a48/users/946c3aca55f243fcac41d96e819fd28d/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.678205 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a3a45-0235-4349-80f0-9b4cf93fbae1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.692176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a3a45-0235-4349-80f0-9b4cf93fbae1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.694874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a3be7-dec0-4ca1-8dc9-b7bdddbdb755 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bbc4217ea2148108dc2165708dc9a48, user_id=946c3aca55f243fcac41d96e819fd28d, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.697438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402f18a-a439-4a62-9a19-f97db0fd9201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.698065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402f18a-a439-4a62-9a19-f97db0fd9201 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.698596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402f18a-a439-4a62-9a19-f97db0fd9201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.698746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a3be7-dec0-4ca1-8dc9-b7bdddbdb755 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.698890 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402f18a-a439-4a62-9a19-f97db0fd9201 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.699535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402f18a-a439-4a62-9a19-f97db0fd9201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:19:55.699870 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402f18a-a439-4a62-9a19-f97db0fd9201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.701653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402f18a-a439-4a62-9a19-f97db0fd9201 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.716128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1021/2037] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:19:55.721276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402f18a-a439-4a62-9a19-f97db0fd9201 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.735064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402f18a-a439-4a62-9a19-f97db0fd9201 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:55.736424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1017/2038] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:55 2022] PUT /identity/v3/projects/6bbc4217ea2148108dc2165708dc9a48/users/946c3aca55f243fcac41d96e819fd28d/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:55.747210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239ecfd-ca58-4703-bd47-2cde9d89a601 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.747625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239ecfd-ca58-4703-bd47-2cde9d89a601 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.749258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239ecfd-ca58-4703-bd47-2cde9d89a601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.754710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:55.810082 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7239ecfd-ca58-4703-bd47-2cde9d89a601 None None] MFA Rules not processed for user `946c3aca55f243fcac41d96e819fd28d`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.811834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7239ecfd-ca58-4703-bd47-2cde9d89a601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.822875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf021dae-8b96-4b1a-b2c6-393219673591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:55.823917 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf021dae-8b96-4b1a-b2c6-393219673591 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.823917 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf021dae-8b96-4b1a-b2c6-393219673591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.824385 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf021dae-8b96-4b1a-b2c6-393219673591 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.825181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf021dae-8b96-4b1a-b2c6-393219673591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:55.827275 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf021dae-8b96-4b1a-b2c6-393219673591 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:55.841961 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:55.890112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1022/2039] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/projects => generated 367 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:55.891085 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf021dae-8b96-4b1a-b2c6-393219673591 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:55.923457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1018/2040] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:19:55.928228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905ca3d-eb14-4aab-9bb3-d39fa565ecec None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:55.928514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905ca3d-eb14-4aab-9bb3-d39fa565ecec None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:55.928820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905ca3d-eb14-4aab-9bb3-d39fa565ecec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:55.960529 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a905ca3d-eb14-4aab-9bb3-d39fa565ecec None None] MFA Rules not processed for user `159528b6ad49481f86bfbbac6187d584`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:55.962654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a905ca3d-eb14-4aab-9bb3-d39fa565ecec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:55.989552 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf021dae-8b96-4b1a-b2c6-393219673591 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.001279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed3275-51f7-41a7-8117-4b1fc00bcfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.002801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed3275-51f7-41a7-8117-4b1fc00bcfb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.003478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed3275-51f7-41a7-8117-4b1fc00bcfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.004257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed3275-51f7-41a7-8117-4b1fc00bcfb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.005892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed3275-51f7-41a7-8117-4b1fc00bcfb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.008473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed3275-51f7-41a7-8117-4b1fc00bcfb3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.056725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1019/2041] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:56.057450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905ca3d-eb14-4aab-9bb3-d39fa565ecec None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.078148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905ca3d-eb14-4aab-9bb3-d39fa565ecec None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.078945 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1023/2042] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:55 2022] POST /identity/v3/users => generated 284 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:56.080407 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed3275-51f7-41a7-8117-4b1fc00bcfb3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.083144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-51ed3275-51f7-41a7-8117-4b1fc00bcfb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:56.086372 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9cc1b-4762-4f87-9b70-654a6788e006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.087007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9cc1b-4762-4f87-9b70-654a6788e006 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.087282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9cc1b-4762-4f87-9b70-654a6788e006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.087550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9cc1b-4762-4f87-9b70-654a6788e006 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.088195 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9cc1b-4762-4f87-9b70-654a6788e006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.091571 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9cc1b-4762-4f87-9b70-654a6788e006 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.132084 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1020/2043] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/projects => generated 377 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:56.132943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9cc1b-4762-4f87-9b70-654a6788e006 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.158282 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed3275-51f7-41a7-8117-4b1fc00bcfb3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.202543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9cc1b-4762-4f87-9b70-654a6788e006 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.208360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed27af0-283a-4cde-bcd1-056908415f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.209350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed27af0-283a-4cde-bcd1-056908415f1b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.209629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed27af0-283a-4cde-bcd1-056908415f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.209896 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed27af0-283a-4cde-bcd1-056908415f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.210720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed27af0-283a-4cde-bcd1-056908415f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.212513 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed27af0-283a-4cde-bcd1-056908415f1b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.214444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:56.229281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1021/2044] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:56.230282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed27af0-283a-4cde-bcd1-056908415f1b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.231538 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08dbaa-b377-400f-a5e4-29b3331f5740 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.232420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08dbaa-b377-400f-a5e4-29b3331f5740 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.232800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08dbaa-b377-400f-a5e4-29b3331f5740 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.233172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08dbaa-b377-400f-a5e4-29b3331f5740 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.243551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed27af0-283a-4cde-bcd1-056908415f1b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.249261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f1be6-9378-4a8a-94f5-64cc04699a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.250187 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f1be6-9378-4a8a-94f5-64cc04699a92 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.250654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f1be6-9378-4a8a-94f5-64cc04699a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.250857 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f1be6-9378-4a8a-94f5-64cc04699a92 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.251513 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f1be6-9378-4a8a-94f5-64cc04699a92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.254190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f1be6-9378-4a8a-94f5-64cc04699a92 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.280670 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1022/2045] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/users => generated 295 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:56.281906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f1be6-9378-4a8a-94f5-64cc04699a92 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.294292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f1be6-9378-4a8a-94f5-64cc04699a92 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.300053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08dbaa-b377-400f-a5e4-29b3331f5740 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.300968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557029ea-8273-4075-841e-a62fa67a8431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.301590 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557029ea-8273-4075-841e-a62fa67a8431 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.301767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08dbaa-b377-400f-a5e4-29b3331f5740 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.301938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557029ea-8273-4075-841e-a62fa67a8431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.302496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557029ea-8273-4075-841e-a62fa67a8431 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.303131 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557029ea-8273-4075-841e-a62fa67a8431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.305033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557029ea-8273-4075-841e-a62fa67a8431 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.311728 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1024/2046] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:56.312381 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:56.313320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08dbaa-b377-400f-a5e4-29b3331f5740 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.324255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1023/2047] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:56.325540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557029ea-8273-4075-841e-a62fa67a8431 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.328189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08dbaa-b377-400f-a5e4-29b3331f5740 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.333565 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d64f17-64a3-4ac1-aaf9-9d23efbd89d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.334385 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d64f17-64a3-4ac1-aaf9-9d23efbd89d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.334498 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d64f17-64a3-4ac1-aaf9-9d23efbd89d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.334737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d64f17-64a3-4ac1-aaf9-9d23efbd89d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.336031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d64f17-64a3-4ac1-aaf9-9d23efbd89d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.337066 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d64f17-64a3-4ac1-aaf9-9d23efbd89d7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.339290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557029ea-8273-4075-841e-a62fa67a8431 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.346244 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8fd1a7-bfc9-4ecf-86dd-4402d552098d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.346930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fd1a7-bfc9-4ecf-86dd-4402d552098d admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.347207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fd1a7-bfc9-4ecf-86dd-4402d552098d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.347478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fd1a7-bfc9-4ecf-86dd-4402d552098d admin admin] PATH_INFO: `/v3/projects/9e8ab37d0717488290ef06fe7c86171c/users/534d3e206ab14fef84b9f19c465f2b7a/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.349473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1025/2048] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:56.353474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d64f17-64a3-4ac1-aaf9-9d23efbd89d7 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.365483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d64f17-64a3-4ac1-aaf9-9d23efbd89d7 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.365715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8fd1a7-bfc9-4ecf-86dd-4402d552098d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e8ab37d0717488290ef06fe7c86171c, user_id=534d3e206ab14fef84b9f19c465f2b7a, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.368616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8fd1a7-bfc9-4ecf-86dd-4402d552098d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.371431 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742fcb5-7701-46a9-8e01-286cb35e6fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.372388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742fcb5-7701-46a9-8e01-286cb35e6fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.372725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742fcb5-7701-46a9-8e01-286cb35e6fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.373607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742fcb5-7701-46a9-8e01-286cb35e6fb1 admin admin] PATH_INFO: `/v3/projects/896051e516ed49f6bfabcf3b0711b000/users/73c7875f4aa4428f9d3c4faa4effdc5c/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.395156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3742fcb5-7701-46a9-8e01-286cb35e6fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896051e516ed49f6bfabcf3b0711b000, user_id=73c7875f4aa4428f9d3c4faa4effdc5c, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.399610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3742fcb5-7701-46a9-8e01-286cb35e6fb1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.407278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1024/2049] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:56 2022] PUT /identity/v3/projects/9e8ab37d0717488290ef06fe7c86171c/users/534d3e206ab14fef84b9f19c465f2b7a/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:56.420064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32fc9f-ca52-4895-b40f-5a7797948f70 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.420470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32fc9f-ca52-4895-b40f-5a7797948f70 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.420777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32fc9f-ca52-4895-b40f-5a7797948f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.446053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1026/2050] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:56 2022] PUT /identity/v3/projects/896051e516ed49f6bfabcf3b0711b000/users/73c7875f4aa4428f9d3c4faa4effdc5c/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:56.450583 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742fcb5-7701-46a9-8e01-286cb35e6fb1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.458031 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0b32fc9f-ca52-4895-b40f-5a7797948f70 None None] MFA Rules not processed for user `534d3e206ab14fef84b9f19c465f2b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:56.459694 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0b32fc9f-ca52-4895-b40f-5a7797948f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:56.552571 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742fcb5-7701-46a9-8e01-286cb35e6fb1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.557843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc63a3-5d7e-4abd-8582-d57d397fba2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.558893 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc63a3-5d7e-4abd-8582-d57d397fba2e admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.558893 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc63a3-5d7e-4abd-8582-d57d397fba2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.559104 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc63a3-5d7e-4abd-8582-d57d397fba2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.559669 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc63a3-5d7e-4abd-8582-d57d397fba2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.561311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc63a3-5d7e-4abd-8582-d57d397fba2e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.577635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1027/2051] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:19:56.581706 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc63a3-5d7e-4abd-8582-d57d397fba2e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.592709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc63a3-5d7e-4abd-8582-d57d397fba2e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.602159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1025/2052] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:19:56.604120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816f9b5-5cd8-4061-9701-2c4290e9a665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.604885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816f9b5-5cd8-4061-9701-2c4290e9a665 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.606188 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816f9b5-5cd8-4061-9701-2c4290e9a665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.606188 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816f9b5-5cd8-4061-9701-2c4290e9a665 admin admin] PATH_INFO: `/v3/projects/896051e516ed49f6bfabcf3b0711b000/users/73c7875f4aa4428f9d3c4faa4effdc5c/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.607984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816f9b5-5cd8-4061-9701-2c4290e9a665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896051e516ed49f6bfabcf3b0711b000, user_id=73c7875f4aa4428f9d3c4faa4effdc5c, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.608400 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32fc9f-ca52-4895-b40f-5a7797948f70 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.611703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816f9b5-5cd8-4061-9701-2c4290e9a665 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.636639 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1028/2053] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:56 2022] PUT /identity/v3/projects/896051e516ed49f6bfabcf3b0711b000/users/73c7875f4aa4428f9d3c4faa4effdc5c/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:19:56.648224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d1807-ed03-4c08-ba36-7b7b72f8e1d6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.648792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d1807-ed03-4c08-ba36-7b7b72f8e1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.649159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d1807-ed03-4c08-ba36-7b7b72f8e1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.650220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32fc9f-ca52-4895-b40f-5a7797948f70 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.709231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b23d1807-ed03-4c08-ba36-7b7b72f8e1d6 None None] MFA Rules not processed for user `73c7875f4aa4428f9d3c4faa4effdc5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:56.710483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b23d1807-ed03-4c08-ba36-7b7b72f8e1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:56.793909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:56.801489 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306999fa-8f88-492d-acec-de85d6011311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.809549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306999fa-8f88-492d-acec-de85d6011311 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.809862 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306999fa-8f88-492d-acec-de85d6011311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.810140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306999fa-8f88-492d-acec-de85d6011311 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.810949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306999fa-8f88-492d-acec-de85d6011311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.813134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306999fa-8f88-492d-acec-de85d6011311 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:56.863478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1029/2054] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:56.879967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1026/2055] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/projects => generated 369 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:56.884903 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d1807-ed03-4c08-ba36-7b7b72f8e1d6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.903068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d1807-ed03-4c08-ba36-7b7b72f8e1d6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:56.945838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306999fa-8f88-492d-acec-de85d6011311 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:56.970645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcadd7-67fd-4252-839a-dde0b4180065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:56.971342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcadd7-67fd-4252-839a-dde0b4180065 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:56.972023 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcadd7-67fd-4252-839a-dde0b4180065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:56.972329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcadd7-67fd-4252-839a-dde0b4180065 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:56.973113 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcadd7-67fd-4252-839a-dde0b4180065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:56.975045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcadd7-67fd-4252-839a-dde0b4180065 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.000648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1030/2056] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:19:56 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:19:57.003336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcadd7-67fd-4252-839a-dde0b4180065 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.008689 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306999fa-8f88-492d-acec-de85d6011311 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.012250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77667e35-697b-44c1-825e-f2feba4168a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.012883 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77667e35-697b-44c1-825e-f2feba4168a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.013081 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcadd7-67fd-4252-839a-dde0b4180065 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.013223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77667e35-697b-44c1-825e-f2feba4168a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.013441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77667e35-697b-44c1-825e-f2feba4168a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.015172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-77667e35-697b-44c1-825e-f2feba4168a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:57.018104 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1c350-3a9b-424d-8734-3e45308ae576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.018694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c350-3a9b-424d-8734-3e45308ae576 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.018968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c350-3a9b-424d-8734-3e45308ae576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.019235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c350-3a9b-424d-8734-3e45308ae576 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.019867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c350-3a9b-424d-8734-3e45308ae576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.020472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:57.021480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c350-3a9b-424d-8734-3e45308ae576 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.032315 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1031/2057] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:57.037063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1c350-3a9b-424d-8734-3e45308ae576 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.047152 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1c350-3a9b-424d-8734-3e45308ae576 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.051819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b932d-73aa-40d2-958f-4d5d4d614323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.052466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b932d-73aa-40d2-958f-4d5d4d614323 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.052791 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b932d-73aa-40d2-958f-4d5d4d614323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.053061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b932d-73aa-40d2-958f-4d5d4d614323 admin admin] PATH_INFO: `/v3/projects/64ee4a7d8c6b4830bcd43a3b332dac13/users/f3a405ffe333410aafadc0cc12cec82c/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.067632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b932d-73aa-40d2-958f-4d5d4d614323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ee4a7d8c6b4830bcd43a3b332dac13, user_id=f3a405ffe333410aafadc0cc12cec82c, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.070339 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b932d-73aa-40d2-958f-4d5d4d614323 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.075768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77667e35-697b-44c1-825e-f2feba4168a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.077241 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77667e35-697b-44c1-825e-f2feba4168a8 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.084755 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1032/2058] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:57 2022] PUT /identity/v3/projects/64ee4a7d8c6b4830bcd43a3b332dac13/users/f3a405ffe333410aafadc0cc12cec82c/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:57.087434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1027/2059] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:57.088763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b932d-73aa-40d2-958f-4d5d4d614323 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.152139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b932d-73aa-40d2-958f-4d5d4d614323 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.157364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4695d8-4aba-4182-9c5f-6ef8e46243a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.157879 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4695d8-4aba-4182-9c5f-6ef8e46243a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.158154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4695d8-4aba-4182-9c5f-6ef8e46243a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.158515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4695d8-4aba-4182-9c5f-6ef8e46243a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.159225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4695d8-4aba-4182-9c5f-6ef8e46243a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.161027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4695d8-4aba-4182-9c5f-6ef8e46243a7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.175898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1033/2060] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:19:57.177436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77667e35-697b-44c1-825e-f2feba4168a8 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.188406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77667e35-697b-44c1-825e-f2feba4168a8 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.193633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098851a-010f-423d-b149-ff57c2194d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.194400 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098851a-010f-423d-b149-ff57c2194d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.194739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098851a-010f-423d-b149-ff57c2194d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.195075 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098851a-010f-423d-b149-ff57c2194d4c admin admin] PATH_INFO: `/v3/projects/64ee4a7d8c6b4830bcd43a3b332dac13/users/f3a405ffe333410aafadc0cc12cec82c/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.197139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098851a-010f-423d-b149-ff57c2194d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ee4a7d8c6b4830bcd43a3b332dac13, user_id=f3a405ffe333410aafadc0cc12cec82c, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.199854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098851a-010f-423d-b149-ff57c2194d4c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.210917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1028/2061] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:19:57 2022] PUT /identity/v3/projects/64ee4a7d8c6b4830bcd43a3b332dac13/users/f3a405ffe333410aafadc0cc12cec82c/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:57.222355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22357bd-54e9-45ac-8326-df9fe0b62c60 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.222355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22357bd-54e9-45ac-8326-df9fe0b62c60 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.222632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22357bd-54e9-45ac-8326-df9fe0b62c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.253554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a22357bd-54e9-45ac-8326-df9fe0b62c60 None None] MFA Rules not processed for user `f3a405ffe333410aafadc0cc12cec82c`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:57.254777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a22357bd-54e9-45ac-8326-df9fe0b62c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:57.264219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:57.337848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1034/2062] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:19:57.348679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098851a-010f-423d-b149-ff57c2194d4c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:57.415849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098851a-010f-423d-b149-ff57c2194d4c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:57.420666 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e528b-b655-4298-af28-673122cfd221 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:57.421475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e528b-b655-4298-af28-673122cfd221 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:57.421822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e528b-b655-4298-af28-673122cfd221 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:57.422785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e528b-b655-4298-af28-673122cfd221 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:57.442730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e528b-b655-4298-af28-673122cfd221 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:57.444478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e528b-b655-4298-af28-673122cfd221 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:57.454754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1029/2063] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:58.050805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c10b2-1dc1-4d4a-ae3a-b22a4c970750 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:58.051568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c10b2-1dc1-4d4a-ae3a-b22a4c970750 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:58.051568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c10b2-1dc1-4d4a-ae3a-b22a4c970750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:58.072004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-667c10b2-1dc1-4d4a-ae3a-b22a4c970750 None None] MFA Rules not processed for user `159528b6ad49481f86bfbbac6187d584`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:58.073226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-667c10b2-1dc1-4d4a-ae3a-b22a4c970750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:58.136725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1035/2064] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:59.075637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e528b-b655-4298-af28-673122cfd221 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.196826 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e528b-b655-4298-af28-673122cfd221 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.202490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.203335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.203585 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.203950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] PATH_INFO: `/v3/users/534d3e206ab14fef84b9f19c465f2b7a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.205550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=534d3e206ab14fef84b9f19c465f2b7a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.208435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.269709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:59.282070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '534d3e206ab14fef84b9f19c465f2b7a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:59.282996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '534d3e206ab14fef84b9f19c465f2b7a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:59.290286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '534d3e206ab14fef84b9f19c465f2b7a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:59.302776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '534d3e206ab14fef84b9f19c465f2b7a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:59.303628 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '534d3e206ab14fef84b9f19c465f2b7a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:19:59.319261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1030/2065] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:19:59 2022] DELETE /identity/v3/users/534d3e206ab14fef84b9f19c465f2b7a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:19:59.324049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf9a5f-8bf1-4ee4-b602-5534f62b1b9a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.324572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf9a5f-8bf1-4ee4-b602-5534f62b1b9a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.324912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf9a5f-8bf1-4ee4-b602-5534f62b1b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.355935 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-07cf9a5f-8bf1-4ee4-b602-5534f62b1b9a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:19:59.357169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-07cf9a5f-8bf1-4ee4-b602-5534f62b1b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:59.435260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1036/2066] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:19:59.446443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.448136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:59.548327 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039aef2-8566-4d67-9915-ca908fee1328 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.554654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9281d255-2fed-4046-b23f-35c833d547f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.554654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281d255-2fed-4046-b23f-35c833d547f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.554654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281d255-2fed-4046-b23f-35c833d547f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.554654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9281d255-2fed-4046-b23f-35c833d547f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.566300 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9281d255-2fed-4046-b23f-35c833d547f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.567754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9281d255-2fed-4046-b23f-35c833d547f7 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.579697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1031/2067] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:19:59.774723 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf9a5f-8bf1-4ee4-b602-5534f62b1b9a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.776704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-07cf9a5f-8bf1-4ee4-b602-5534f62b1b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:59.851756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf9a5f-8bf1-4ee4-b602-5534f62b1b9a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.857074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a1be7-9e19-420e-9cf1-e271419398d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.857941 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a1be7-9e19-420e-9cf1-e271419398d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.858224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a1be7-9e19-420e-9cf1-e271419398d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.858849 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a1be7-9e19-420e-9cf1-e271419398d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.860444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-842a1be7-9e19-420e-9cf1-e271419398d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:59.870968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:19:59.932427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9281d255-2fed-4046-b23f-35c833d547f7 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:19:59.934064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9281d255-2fed-4046-b23f-35c833d547f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:19:59.943555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a1be7-9e19-420e-9cf1-e271419398d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.945274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a1be7-9e19-420e-9cf1-e271419398d5 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:19:59.954926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1037/2068] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:19:59.964904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9281d255-2fed-4046-b23f-35c833d547f7 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:19:59.970980 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:19:59.971929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:19:59.972114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:19:59.972389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] PATH_INFO: `/v3/projects/9e8ab37d0717488290ef06fe7c86171c` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:19:59.974560 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e8ab37d0717488290ef06fe7c86171c)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:19:59.975942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:00.030258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e8ab37d0717488290ef06fe7c86171c', 'request_id': 'req-15334b79-8d22-440b-981c-7dc412aa190b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:00.041435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e8ab37d0717488290ef06fe7c86171c', 'request_id': 'req-15334b79-8d22-440b-981c-7dc412aa190b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:00.055144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e8ab37d0717488290ef06fe7c86171c', 'request_id': 'req-15334b79-8d22-440b-981c-7dc412aa190b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:00.073191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] The token cache is being invalidate because project 9e8ab37d0717488290ef06fe7c86171c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:00.073499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:00.074555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:00.076860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1032/2069] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:19:59 2022] DELETE /identity/v3/projects/9e8ab37d0717488290ef06fe7c86171c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:00.081575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a1be7-9e19-420e-9cf1-e271419398d5 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:00.093064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a1be7-9e19-420e-9cf1-e271419398d5 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:00.160449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:00.161503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:00.161812 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:00.162086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] PATH_INFO: `/v3/users/f3a405ffe333410aafadc0cc12cec82c` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:00.163286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a405ffe333410aafadc0cc12cec82c)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:00.165236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:00.214897 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a405ffe333410aafadc0cc12cec82c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:00.222431 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a405ffe333410aafadc0cc12cec82c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:00.236260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a405ffe333410aafadc0cc12cec82c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:00.245766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a405ffe333410aafadc0cc12cec82c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:00.246400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a405ffe333410aafadc0cc12cec82c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:00.253694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1038/2070] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:00 2022] DELETE /identity/v3/users/f3a405ffe333410aafadc0cc12cec82c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:00.709375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:00.710973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:00.769514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:00.787957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15334b79-8d22-440b-981c-7dc412aa190b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:00.800338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:00.802045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:00.845547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:00.852259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241153c-c498-4d99-9ed1-c9114b1b6c20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:00.853042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241153c-c498-4d99-9ed1-c9114b1b6c20 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:00.853344 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241153c-c498-4d99-9ed1-c9114b1b6c20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:00.854076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241153c-c498-4d99-9ed1-c9114b1b6c20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:00.855992 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6241153c-c498-4d99-9ed1-c9114b1b6c20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:00.900766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ecf20-527a-419f-a00d-46ece5543334 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:00.909588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d69f03-534f-47ae-b4f1-555cc01da49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:00.910244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d69f03-534f-47ae-b4f1-555cc01da49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:00.910873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d69f03-534f-47ae-b4f1-555cc01da49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:00.911141 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d69f03-534f-47ae-b4f1-555cc01da49a admin admin] PATH_INFO: `/v3/projects/64ee4a7d8c6b4830bcd43a3b332dac13` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:00.912535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d69f03-534f-47ae-b4f1-555cc01da49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=64ee4a7d8c6b4830bcd43a3b332dac13)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:00.915772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d69f03-534f-47ae-b4f1-555cc01da49a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:00.940981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241153c-c498-4d99-9ed1-c9114b1b6c20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:00.943495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241153c-c498-4d99-9ed1-c9114b1b6c20 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:00.956450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1033/2071] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:00.981404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49d69f03-534f-47ae-b4f1-555cc01da49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64ee4a7d8c6b4830bcd43a3b332dac13', 'request_id': 'req-49d69f03-534f-47ae-b4f1-555cc01da49a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:00.993009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49d69f03-534f-47ae-b4f1-555cc01da49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64ee4a7d8c6b4830bcd43a3b332dac13', 'request_id': 'req-49d69f03-534f-47ae-b4f1-555cc01da49a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:01.000468 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49d69f03-534f-47ae-b4f1-555cc01da49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64ee4a7d8c6b4830bcd43a3b332dac13', 'request_id': 'req-49d69f03-534f-47ae-b4f1-555cc01da49a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:01.027247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49d69f03-534f-47ae-b4f1-555cc01da49a admin admin] The token cache is being invalidate because project 64ee4a7d8c6b4830bcd43a3b332dac13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:01.028032 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49d69f03-534f-47ae-b4f1-555cc01da49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:01.042859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49d69f03-534f-47ae-b4f1-555cc01da49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:01.046203 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1039/2072] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:00 2022] DELETE /identity/v3/projects/64ee4a7d8c6b4830bcd43a3b332dac13 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:01.051266 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978133cd-6a14-4f36-be78-8213d216db21 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:01.051591 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978133cd-6a14-4f36-be78-8213d216db21 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:01.051959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978133cd-6a14-4f36-be78-8213d216db21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:01.076988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-978133cd-6a14-4f36-be78-8213d216db21 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:01.078363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-978133cd-6a14-4f36-be78-8213d216db21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:01.183966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1034/2073] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:01.197676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871b074-8bee-4e17-9587-a8bf22bb2a5b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:01.197958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871b074-8bee-4e17-9587-a8bf22bb2a5b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:01.198234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871b074-8bee-4e17-9587-a8bf22bb2a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:01.219055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c871b074-8bee-4e17-9587-a8bf22bb2a5b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:01.220376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c871b074-8bee-4e17-9587-a8bf22bb2a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:01.229381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1040/2074] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:01.233690 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978133cd-6a14-4f36-be78-8213d216db21 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:01.247806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978133cd-6a14-4f36-be78-8213d216db21 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:01.254235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ff791-3e3d-4126-b414-8e297c2849be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:01.254950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ff791-3e3d-4126-b414-8e297c2849be admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:01.255280 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ff791-3e3d-4126-b414-8e297c2849be admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:01.255606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ff791-3e3d-4126-b414-8e297c2849be admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:01.256320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ff791-3e3d-4126-b414-8e297c2849be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:01.256711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ff791-3e3d-4126-b414-8e297c2849be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:01.258508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ff791-3e3d-4126-b414-8e297c2849be admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:01.272861 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1035/2075] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:20:01.277433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871b074-8bee-4e17-9587-a8bf22bb2a5b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:01.289013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871b074-8bee-4e17-9587-a8bf22bb2a5b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:01.295263 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14722671-0131-4e83-8646-53b7defc38fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:01.296200 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14722671-0131-4e83-8646-53b7defc38fc admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:01.296647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14722671-0131-4e83-8646-53b7defc38fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:01.297100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14722671-0131-4e83-8646-53b7defc38fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:01.298068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14722671-0131-4e83-8646-53b7defc38fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:01.300916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14722671-0131-4e83-8646-53b7defc38fc admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:01.336881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1041/2076] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:01 2022] POST /identity/v3/projects => generated 409 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:01.341222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ff791-3e3d-4126-b414-8e297c2849be admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:01.356481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ff791-3e3d-4126-b414-8e297c2849be admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:01.403146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:01.441779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db9bce-3a76-4d97-9d99-279fba24bc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:01.442522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db9bce-3a76-4d97-9d99-279fba24bc97 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:01.442684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db9bce-3a76-4d97-9d99-279fba24bc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:01.443583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db9bce-3a76-4d97-9d99-279fba24bc97 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:01.444276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db9bce-3a76-4d97-9d99-279fba24bc97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:01.446273 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db9bce-3a76-4d97-9d99-279fba24bc97 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:01.472776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1036/2077] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:01 2022] POST /identity/v3/users => generated 305 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:20:01.474116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14722671-0131-4e83-8646-53b7defc38fc admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:01.488433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14722671-0131-4e83-8646-53b7defc38fc admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:01.494422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a28ae-ba96-4d59-bc3f-3a9e2115da7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:01.495228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a28ae-ba96-4d59-bc3f-3a9e2115da7f admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:01.495506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a28ae-ba96-4d59-bc3f-3a9e2115da7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:01.495774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a28ae-ba96-4d59-bc3f-3a9e2115da7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:01.496685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a28ae-ba96-4d59-bc3f-3a9e2115da7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:01.500361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a28ae-ba96-4d59-bc3f-3a9e2115da7f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:01.514540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1042/2078] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:20:01.522361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db9bce-3a76-4d97-9d99-279fba24bc97 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:01.554903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db9bce-3a76-4d97-9d99-279fba24bc97 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:01.561469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435a748-418e-4348-aa33-a130d7902617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:01.562620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435a748-418e-4348-aa33-a130d7902617 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:01.562974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435a748-418e-4348-aa33-a130d7902617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:01.563345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435a748-418e-4348-aa33-a130d7902617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:01.564085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435a748-418e-4348-aa33-a130d7902617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:01.566346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435a748-418e-4348-aa33-a130d7902617 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:01.579217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1037/2079] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:01.583925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a28ae-ba96-4d59-bc3f-3a9e2115da7f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:01.594193 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a28ae-ba96-4d59-bc3f-3a9e2115da7f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:01.599022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd5408-519e-4f2a-aabe-338f62549ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:01.599697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd5408-519e-4f2a-aabe-338f62549ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:01.599968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd5408-519e-4f2a-aabe-338f62549ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:01.600238 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd5408-519e-4f2a-aabe-338f62549ebc admin admin] PATH_INFO: `/v3/projects/1ecaa6d3fba4403c85276ec63d80dd34/users/0df1bd200e744439b2363168f8687af5/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:01.617973 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd5408-519e-4f2a-aabe-338f62549ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ecaa6d3fba4403c85276ec63d80dd34, user_id=0df1bd200e744439b2363168f8687af5, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:01.621318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd5408-519e-4f2a-aabe-338f62549ebc admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:01.648291 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1043/2080] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:01 2022] PUT /identity/v3/projects/1ecaa6d3fba4403c85276ec63d80dd34/users/0df1bd200e744439b2363168f8687af5/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:01.660065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e608d77-3902-477f-8c4a-8ce100c5f923 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:01.660863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e608d77-3902-477f-8c4a-8ce100c5f923 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:01.661368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e608d77-3902-477f-8c4a-8ce100c5f923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:01.701079 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0e608d77-3902-477f-8c4a-8ce100c5f923 None None] MFA Rules not processed for user `0df1bd200e744439b2363168f8687af5`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:01.702751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0e608d77-3902-477f-8c4a-8ce100c5f923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:01.783779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1038/2081] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:01 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:01.789972 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd5408-519e-4f2a-aabe-338f62549ebc admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:01.857460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:01.872420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd5408-519e-4f2a-aabe-338f62549ebc admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:01.879495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b9c03-4018-479b-95ae-d89534167108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:01.880964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b9c03-4018-479b-95ae-d89534167108 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:01.881386 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b9c03-4018-479b-95ae-d89534167108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:01.881991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b9c03-4018-479b-95ae-d89534167108 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:01.883423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b9c03-4018-479b-95ae-d89534167108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:01.885798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b9c03-4018-479b-95ae-d89534167108 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:01.924046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1044/2082] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:01 2022] POST /identity/v3/projects => generated 409 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:01.927984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e608d77-3902-477f-8c4a-8ce100c5f923 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:01.946043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e608d77-3902-477f-8c4a-8ce100c5f923 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:02.014173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be3325-51e0-4c29-8755-b0c9f5120b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:02.014792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be3325-51e0-4c29-8755-b0c9f5120b63 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:02.015071 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be3325-51e0-4c29-8755-b0c9f5120b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:02.015325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be3325-51e0-4c29-8755-b0c9f5120b63 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:02.015979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be3325-51e0-4c29-8755-b0c9f5120b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:02.018356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be3325-51e0-4c29-8755-b0c9f5120b63 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:02.042449 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1039/2083] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:01 2022] POST /identity/v3/users => generated 311 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:20:02.047036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b9c03-4018-479b-95ae-d89534167108 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:02.067218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b9c03-4018-479b-95ae-d89534167108 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:02.076796 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfeb3fb-a5c5-4774-a552-a72de1b92ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:02.077634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfeb3fb-a5c5-4774-a552-a72de1b92ddb admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:02.078005 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfeb3fb-a5c5-4774-a552-a72de1b92ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:02.078379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfeb3fb-a5c5-4774-a552-a72de1b92ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:02.079335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfeb3fb-a5c5-4774-a552-a72de1b92ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:02.081907 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfeb3fb-a5c5-4774-a552-a72de1b92ddb admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:02.109651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1045/2084] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:02.114353 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be3325-51e0-4c29-8755-b0c9f5120b63 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:02.126855 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be3325-51e0-4c29-8755-b0c9f5120b63 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:02.150985 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387d483-fceb-4344-b312-172f3cae2430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:02.151917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387d483-fceb-4344-b312-172f3cae2430 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:02.152405 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387d483-fceb-4344-b312-172f3cae2430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:02.155436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387d483-fceb-4344-b312-172f3cae2430 admin admin] PATH_INFO: `/v3/projects/11d9008edaad44db8ad515f79e479554/users/290df346f2614738b980ed1d26159a70/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:02.193174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387d483-fceb-4344-b312-172f3cae2430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d9008edaad44db8ad515f79e479554, user_id=290df346f2614738b980ed1d26159a70, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:02.199945 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387d483-fceb-4344-b312-172f3cae2430 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:02.225134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1040/2085] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:02 2022] PUT /identity/v3/projects/11d9008edaad44db8ad515f79e479554/users/290df346f2614738b980ed1d26159a70/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:02.230384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfeb3fb-a5c5-4774-a552-a72de1b92ddb admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:02.261762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfeb3fb-a5c5-4774-a552-a72de1b92ddb admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:02.384380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f703615-8c89-4539-8683-2bc8f1274cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:02.386341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f703615-8c89-4539-8683-2bc8f1274cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:02.386627 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f703615-8c89-4539-8683-2bc8f1274cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:02.386904 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f703615-8c89-4539-8683-2bc8f1274cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:02.387576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f703615-8c89-4539-8683-2bc8f1274cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:02.389386 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f703615-8c89-4539-8683-2bc8f1274cc8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:02.413470 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1046/2086] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:02.417537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387d483-fceb-4344-b312-172f3cae2430 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:02.428916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387d483-fceb-4344-b312-172f3cae2430 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:02.449043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df5c85-094b-42bf-b50c-f7b83e934493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:02.450134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df5c85-094b-42bf-b50c-f7b83e934493 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:02.450698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df5c85-094b-42bf-b50c-f7b83e934493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:02.451284 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df5c85-094b-42bf-b50c-f7b83e934493 admin admin] PATH_INFO: `/v3/projects/11d9008edaad44db8ad515f79e479554/users/290df346f2614738b980ed1d26159a70/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:02.456390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df5c85-094b-42bf-b50c-f7b83e934493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d9008edaad44db8ad515f79e479554, user_id=290df346f2614738b980ed1d26159a70, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:02.459879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df5c85-094b-42bf-b50c-f7b83e934493 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:02.463171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:02.502617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88413f-c160-4ccb-bcc2-05a53c3d2406 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:02.502910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88413f-c160-4ccb-bcc2-05a53c3d2406 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:02.503162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88413f-c160-4ccb-bcc2-05a53c3d2406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:02.509012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1041/2087] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:02 2022] PUT /identity/v3/projects/11d9008edaad44db8ad515f79e479554/users/290df346f2614738b980ed1d26159a70/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:02.558525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9a88413f-c160-4ccb-bcc2-05a53c3d2406 None None] MFA Rules not processed for user `290df346f2614738b980ed1d26159a70`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:02.559750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9a88413f-c160-4ccb-bcc2-05a53c3d2406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:02.657530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1047/2088] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:02.668216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df5c85-094b-42bf-b50c-f7b83e934493 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:02.670479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-18df5c85-094b-42bf-b50c-f7b83e934493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:02.768512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df5c85-094b-42bf-b50c-f7b83e934493 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:02.772532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4739210-b3ab-4139-baa4-ff8d410a8075 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:02.773279 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4739210-b3ab-4139-baa4-ff8d410a8075 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:02.773662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4739210-b3ab-4139-baa4-ff8d410a8075 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:02.773999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4739210-b3ab-4139-baa4-ff8d410a8075 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:02.785427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4739210-b3ab-4139-baa4-ff8d410a8075 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:02.787919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4739210-b3ab-4139-baa4-ff8d410a8075 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:02.800049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1042/2089] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:02 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:03.003199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88413f-c160-4ccb-bcc2-05a53c3d2406 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:03.015784 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88413f-c160-4ccb-bcc2-05a53c3d2406 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:03.020660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda8d9f-e6e0-49d5-880e-3a1359e12886 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:03.021361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda8d9f-e6e0-49d5-880e-3a1359e12886 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:03.021734 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda8d9f-e6e0-49d5-880e-3a1359e12886 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:03.022059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda8d9f-e6e0-49d5-880e-3a1359e12886 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:03.096054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:03.115782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda8d9f-e6e0-49d5-880e-3a1359e12886 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:03.117417 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda8d9f-e6e0-49d5-880e-3a1359e12886 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:03.141871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1048/2090] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:03 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:05.000172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4739210-b3ab-4139-baa4-ff8d410a8075 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.002630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d4739210-b3ab-4139-baa4-ff8d410a8075 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:05.077006 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4739210-b3ab-4139-baa4-ff8d410a8075 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.081519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.082161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.082435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.082700 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] PATH_INFO: `/v3/users/a568b6fd614744abaf5ca41fb26c1f13` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.083956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a568b6fd614744abaf5ca41fb26c1f13)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.085912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.136387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a568b6fd614744abaf5ca41fb26c1f13'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.137098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a568b6fd614744abaf5ca41fb26c1f13'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.143968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a568b6fd614744abaf5ca41fb26c1f13'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.155277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a568b6fd614744abaf5ca41fb26c1f13'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.155912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a568b6fd614744abaf5ca41fb26c1f13'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.172842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1043/2091] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:04 2022] DELETE /identity/v3/users/a568b6fd614744abaf5ca41fb26c1f13 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:05.275350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda8d9f-e6e0-49d5-880e-3a1359e12886 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.276824 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fcda8d9f-e6e0-49d5-880e-3a1359e12886 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:05.291054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda8d9f-e6e0-49d5-880e-3a1359e12886 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.350197 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0f4e2-1c52-4b11-b2b9-72b722d53fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.350739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0f4e2-1c52-4b11-b2b9-72b722d53fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.351005 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0f4e2-1c52-4b11-b2b9-72b722d53fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.351264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0f4e2-1c52-4b11-b2b9-72b722d53fb8 admin admin] PATH_INFO: `/v3/users/8ccf6b70a2b845b7ac2547565f04bf56` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.352498 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0f4e2-1c52-4b11-b2b9-72b722d53fb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ccf6b70a2b845b7ac2547565f04bf56)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.354436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa0f4e2-1c52-4b11-b2b9-72b722d53fb8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.366354 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.368179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:05.385235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587985e6-e5cf-4235-9e27-b91d358a3413 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.396383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214afba9-eb4a-421a-bee4-097c39816a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.397190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214afba9-eb4a-421a-bee4-097c39816a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.397633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214afba9-eb4a-421a-bee4-097c39816a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.397903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214afba9-eb4a-421a-bee4-097c39816a0e admin admin] PATH_INFO: `/v3/projects/b0f632ecb8ee4e6fb59978560789f599` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.399462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214afba9-eb4a-421a-bee4-097c39816a0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f632ecb8ee4e6fb59978560789f599)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.401457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214afba9-eb4a-421a-bee4-097c39816a0e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.420462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:05.435150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3fa0f4e2-1c52-4b11-b2b9-72b722d53fb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ccf6b70a2b845b7ac2547565f04bf56'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.444236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3fa0f4e2-1c52-4b11-b2b9-72b722d53fb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ccf6b70a2b845b7ac2547565f04bf56'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.460727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3fa0f4e2-1c52-4b11-b2b9-72b722d53fb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ccf6b70a2b845b7ac2547565f04bf56'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.466884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-214afba9-eb4a-421a-bee4-097c39816a0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f632ecb8ee4e6fb59978560789f599', 'request_id': 'req-214afba9-eb4a-421a-bee4-097c39816a0e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.474143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3fa0f4e2-1c52-4b11-b2b9-72b722d53fb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ccf6b70a2b845b7ac2547565f04bf56'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.475083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3fa0f4e2-1c52-4b11-b2b9-72b722d53fb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ccf6b70a2b845b7ac2547565f04bf56'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.477198 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-214afba9-eb4a-421a-bee4-097c39816a0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f632ecb8ee4e6fb59978560789f599', 'request_id': 'req-214afba9-eb4a-421a-bee4-097c39816a0e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.483991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1049/2092] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:05 2022] DELETE /identity/v3/users/8ccf6b70a2b845b7ac2547565f04bf56 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:05.489356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31b560-b929-4c55-889a-99b3157709d1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.489855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31b560-b929-4c55-889a-99b3157709d1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.490268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31b560-b929-4c55-889a-99b3157709d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.493252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-214afba9-eb4a-421a-bee4-097c39816a0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f632ecb8ee4e6fb59978560789f599', 'request_id': 'req-214afba9-eb4a-421a-bee4-097c39816a0e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.513528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-214afba9-eb4a-421a-bee4-097c39816a0e admin admin] The token cache is being invalidate because project b0f632ecb8ee4e6fb59978560789f599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:05.513732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-9e31b560-b929-4c55-889a-99b3157709d1 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:05.513818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-214afba9-eb4a-421a-bee4-097c39816a0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.514767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-214afba9-eb4a-421a-bee4-097c39816a0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:05.514888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9e31b560-b929-4c55-889a-99b3157709d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:05.517094 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1044/2093] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:05 2022] DELETE /identity/v3/projects/b0f632ecb8ee4e6fb59978560789f599 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:05.534261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761b276-ad8d-4dc1-9d1c-391d29b53e45 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.534769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761b276-ad8d-4dc1-9d1c-391d29b53e45 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.535259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761b276-ad8d-4dc1-9d1c-391d29b53e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.564069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c761b276-ad8d-4dc1-9d1c-391d29b53e45 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:05.565411 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c761b276-ad8d-4dc1-9d1c-391d29b53e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:05.582731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1050/2094] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:05.592528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31b560-b929-4c55-889a-99b3157709d1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.593981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9e31b560-b929-4c55-889a-99b3157709d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:05.604165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:05.624154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1045/2095] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:05.632099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a7a0c-7125-41ec-9c53-8d943d23638c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.632580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a7a0c-7125-41ec-9c53-8d943d23638c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.632924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a7a0c-7125-41ec-9c53-8d943d23638c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.652167 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-760a7a0c-7125-41ec-9c53-8d943d23638c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:05.653386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-760a7a0c-7125-41ec-9c53-8d943d23638c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:05.654682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31b560-b929-4c55-889a-99b3157709d1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.658714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e352dcb-a321-4e99-a751-79a8bf60eb1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.659378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e352dcb-a321-4e99-a751-79a8bf60eb1c service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.659594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e352dcb-a321-4e99-a751-79a8bf60eb1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.659866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e352dcb-a321-4e99-a751-79a8bf60eb1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.661578 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1046/2096] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:05.665602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a7a0c-7125-41ec-9c53-8d943d23638c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.669839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e352dcb-a321-4e99-a751-79a8bf60eb1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.671263 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e352dcb-a321-4e99-a751-79a8bf60eb1c service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.674534 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a7a0c-7125-41ec-9c53-8d943d23638c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.679052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6569423-85cb-4c11-abb8-995aeb3d37e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.679631 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6569423-85cb-4c11-abb8-995aeb3d37e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.679917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6569423-85cb-4c11-abb8-995aeb3d37e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.680157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6569423-85cb-4c11-abb8-995aeb3d37e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.680844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6569423-85cb-4c11-abb8-995aeb3d37e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:05.681037 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1051/2097] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:05.681127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6569423-85cb-4c11-abb8-995aeb3d37e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.682734 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6569423-85cb-4c11-abb8-995aeb3d37e1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.694715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1047/2098] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:20:05.698981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e352dcb-a321-4e99-a751-79a8bf60eb1c service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.708619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e352dcb-a321-4e99-a751-79a8bf60eb1c service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.713848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d93fc-a50f-4817-953e-13f148e85777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.714456 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d93fc-a50f-4817-953e-13f148e85777 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.714753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d93fc-a50f-4817-953e-13f148e85777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.715295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d93fc-a50f-4817-953e-13f148e85777 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.715649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d93fc-a50f-4817-953e-13f148e85777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.717577 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d93fc-a50f-4817-953e-13f148e85777 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.757820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1052/2099] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:05 2022] POST /identity/v3/projects => generated 359 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:20:05.762398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6569423-85cb-4c11-abb8-995aeb3d37e1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.773315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6569423-85cb-4c11-abb8-995aeb3d37e1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.833903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc61000-6d66-408b-b9e6-583854503f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.834521 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc61000-6d66-408b-b9e6-583854503f07 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.834841 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc61000-6d66-408b-b9e6-583854503f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.835108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc61000-6d66-408b-b9e6-583854503f07 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.835962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc61000-6d66-408b-b9e6-583854503f07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.838499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc61000-6d66-408b-b9e6-583854503f07 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.867421 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1048/2100] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:05 2022] POST /identity/v3/users => generated 280 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:05.868934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d93fc-a50f-4817-953e-13f148e85777 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.879515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d93fc-a50f-4817-953e-13f148e85777 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.884567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46a039-57b1-4278-aea2-e41257b07d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.885294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46a039-57b1-4278-aea2-e41257b07d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.885633 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46a039-57b1-4278-aea2-e41257b07d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.885909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46a039-57b1-4278-aea2-e41257b07d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.886561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46a039-57b1-4278-aea2-e41257b07d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.888179 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46a039-57b1-4278-aea2-e41257b07d03 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.898548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1053/2101] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:05.902583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc61000-6d66-408b-b9e6-583854503f07 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.913894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc61000-6d66-408b-b9e6-583854503f07 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.919994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de0a7a-2536-46c3-a694-23d333ed7c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.920669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de0a7a-2536-46c3-a694-23d333ed7c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.920948 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de0a7a-2536-46c3-a694-23d333ed7c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.921210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de0a7a-2536-46c3-a694-23d333ed7c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.921889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de0a7a-2536-46c3-a694-23d333ed7c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.924123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de0a7a-2536-46c3-a694-23d333ed7c58 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.926519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:05.937999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1049/2102] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:05.946086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46a039-57b1-4278-aea2-e41257b07d03 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.956663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46a039-57b1-4278-aea2-e41257b07d03 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.961641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe7bff-b187-4d0c-8620-d1f7c13a48e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.962302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe7bff-b187-4d0c-8620-d1f7c13a48e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.962578 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe7bff-b187-4d0c-8620-d1f7c13a48e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.962844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe7bff-b187-4d0c-8620-d1f7c13a48e7 admin admin] PATH_INFO: `/v3/projects/edd1c2e1025c4053b0d31e1d18482e44/users/1d08531c294f451f898abc384a365d01/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.966836 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de0a7a-2536-46c3-a694-23d333ed7c58 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:05.968505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-38de0a7a-2536-46c3-a694-23d333ed7c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:05.984903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de0a7a-2536-46c3-a694-23d333ed7c58 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:05.991819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe7bff-b187-4d0c-8620-d1f7c13a48e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edd1c2e1025c4053b0d31e1d18482e44, user_id=1d08531c294f451f898abc384a365d01, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:05.994912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:05.995133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fe7bff-b187-4d0c-8620-d1f7c13a48e7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:05.995696 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:05.996035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:05.996375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] PATH_INFO: `/v3/projects/189822c3b45a42da97b87c16b0d2e529` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:05.998006 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] RBAC: Authorizing `identity:delete_project(project_id=189822c3b45a42da97b87c16b0d2e529)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:06.001604 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:06.021227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1054/2103] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:05 2022] PUT /identity/v3/projects/edd1c2e1025c4053b0d31e1d18482e44/users/1d08531c294f451f898abc384a365d01/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:06.050310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61ad60-db57-4c74-a6da-9cb280977865 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:06.050787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61ad60-db57-4c74-a6da-9cb280977865 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:06.051200 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61ad60-db57-4c74-a6da-9cb280977865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:06.080434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1f61ad60-db57-4c74-a6da-9cb280977865 None None] MFA Rules not processed for user `1d08531c294f451f898abc384a365d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:06.081757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1f61ad60-db57-4c74-a6da-9cb280977865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:06.089228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '189822c3b45a42da97b87c16b0d2e529', 'request_id': 'req-ead81cb7-8f22-4721-a258-d9a4bb24944a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.099304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '189822c3b45a42da97b87c16b0d2e529', 'request_id': 'req-ead81cb7-8f22-4721-a258-d9a4bb24944a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.106259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '189822c3b45a42da97b87c16b0d2e529', 'request_id': 'req-ead81cb7-8f22-4721-a258-d9a4bb24944a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.118090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] The token cache is being invalidate because project 189822c3b45a42da97b87c16b0d2e529 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:06.118325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.119246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.121298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1050/2104] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:05 2022] DELETE /identity/v3/projects/189822c3b45a42da97b87c16b0d2e529 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:06.125037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:06.126967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:06.153838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1055/2105] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:06 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:06.159698 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61ad60-db57-4c74-a6da-9cb280977865 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:06.161649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1f61ad60-db57-4c74-a6da-9cb280977865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:06.205486 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead81cb7-8f22-4721-a258-d9a4bb24944a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:06.210548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d2634-d2c1-4da2-914b-29fbbe30a565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:06.211192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d2634-d2c1-4da2-914b-29fbbe30a565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:06.211535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d2634-d2c1-4da2-914b-29fbbe30a565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:06.211875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d2634-d2c1-4da2-914b-29fbbe30a565 admin admin] PATH_INFO: `/v3/users/946c3aca55f243fcac41d96e819fd28d` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:06.213182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d2634-d2c1-4da2-914b-29fbbe30a565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=946c3aca55f243fcac41d96e819fd28d)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:06.215134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:06.215503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d2634-d2c1-4da2-914b-29fbbe30a565 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:06.256158 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61ad60-db57-4c74-a6da-9cb280977865 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:06.261259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90857e9a-75c7-435b-abec-536cb33166be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:06.261985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90857e9a-75c7-435b-abec-536cb33166be admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:06.262612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90857e9a-75c7-435b-abec-536cb33166be admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:06.262947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90857e9a-75c7-435b-abec-536cb33166be admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:06.263659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90857e9a-75c7-435b-abec-536cb33166be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:06.266120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90857e9a-75c7-435b-abec-536cb33166be admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:06.304173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3a7d2634-d2c1-4da2-914b-29fbbe30a565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '946c3aca55f243fcac41d96e819fd28d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.304893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3a7d2634-d2c1-4da2-914b-29fbbe30a565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '946c3aca55f243fcac41d96e819fd28d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.307073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1056/2106] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:06 2022] POST /identity/v3/projects => generated 365 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:20:06.311115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90857e9a-75c7-435b-abec-536cb33166be admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:06.312544 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3a7d2634-d2c1-4da2-914b-29fbbe30a565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '946c3aca55f243fcac41d96e819fd28d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.312735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-90857e9a-75c7-435b-abec-536cb33166be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:06.326583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3a7d2634-d2c1-4da2-914b-29fbbe30a565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '946c3aca55f243fcac41d96e819fd28d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.327278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3a7d2634-d2c1-4da2-914b-29fbbe30a565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '946c3aca55f243fcac41d96e819fd28d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.336944 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:06.342078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1051/2107] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:06 2022] DELETE /identity/v3/users/946c3aca55f243fcac41d96e819fd28d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:06.381973 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90857e9a-75c7-435b-abec-536cb33166be admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:06.442676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b3d03-96b9-4d0a-8754-71dd8c89a6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:06.443366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b3d03-96b9-4d0a-8754-71dd8c89a6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:06.443694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b3d03-96b9-4d0a-8754-71dd8c89a6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:06.443977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b3d03-96b9-4d0a-8754-71dd8c89a6e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:06.444694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b3d03-96b9-4d0a-8754-71dd8c89a6e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:06.446700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b3d03-96b9-4d0a-8754-71dd8c89a6e5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:06.472057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1057/2108] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:06 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:06.473410 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d2634-d2c1-4da2-914b-29fbbe30a565 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:06.483692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d2634-d2c1-4da2-914b-29fbbe30a565 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:06.490220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bc176-5898-47bb-bc74-4b19778b5571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:06.490834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc176-5898-47bb-bc74-4b19778b5571 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:06.491108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc176-5898-47bb-bc74-4b19778b5571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:06.491370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc176-5898-47bb-bc74-4b19778b5571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:06.492020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bc176-5898-47bb-bc74-4b19778b5571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:06.494041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bc176-5898-47bb-bc74-4b19778b5571 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:06.505109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1052/2109] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:06.509565 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b3d03-96b9-4d0a-8754-71dd8c89a6e5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:06.520582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b3d03-96b9-4d0a-8754-71dd8c89a6e5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:06.525501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503292a-eb9a-4d7a-9bd8-2609b5c34c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:06.526162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503292a-eb9a-4d7a-9bd8-2609b5c34c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:06.526573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503292a-eb9a-4d7a-9bd8-2609b5c34c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:06.526844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503292a-eb9a-4d7a-9bd8-2609b5c34c79 admin admin] PATH_INFO: `/v3/projects/3d886cff441f4a54a659a1393fb77a40/users/89a6d355c3174f6ab99ae4d0cdb901ea/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:06.567258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503292a-eb9a-4d7a-9bd8-2609b5c34c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d886cff441f4a54a659a1393fb77a40, user_id=89a6d355c3174f6ab99ae4d0cdb901ea, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:06.570840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503292a-eb9a-4d7a-9bd8-2609b5c34c79 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:06.596830 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1058/2110] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:06 2022] PUT /identity/v3/projects/3d886cff441f4a54a659a1393fb77a40/users/89a6d355c3174f6ab99ae4d0cdb901ea/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:06.607106 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bc176-5898-47bb-bc74-4b19778b5571 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:06.622161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bc176-5898-47bb-bc74-4b19778b5571 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:06.661771 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503292a-eb9a-4d7a-9bd8-2609b5c34c79 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:06.663904 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7503292a-eb9a-4d7a-9bd8-2609b5c34c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:06.695283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523bd8f-2188-48c9-acb6-726c72cf16de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:06.695511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523bd8f-2188-48c9-acb6-726c72cf16de admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:06.695606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523bd8f-2188-48c9-acb6-726c72cf16de admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:06.695851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523bd8f-2188-48c9-acb6-726c72cf16de admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:06.696496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523bd8f-2188-48c9-acb6-726c72cf16de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:06.698147 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523bd8f-2188-48c9-acb6-726c72cf16de admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:06.709064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1053/2111] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:06.713040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523bd8f-2188-48c9-acb6-726c72cf16de admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:06.723779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523bd8f-2188-48c9-acb6-726c72cf16de admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:06.728622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600f3dc-29e4-42e6-a97d-e4966e28fabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:06.729340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600f3dc-29e4-42e6-a97d-e4966e28fabf admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:06.729674 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600f3dc-29e4-42e6-a97d-e4966e28fabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:06.730050 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600f3dc-29e4-42e6-a97d-e4966e28fabf admin admin] PATH_INFO: `/v3/projects/3d886cff441f4a54a659a1393fb77a40/users/89a6d355c3174f6ab99ae4d0cdb901ea/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:06.730522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:06.732325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600f3dc-29e4-42e6-a97d-e4966e28fabf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d886cff441f4a54a659a1393fb77a40, user_id=89a6d355c3174f6ab99ae4d0cdb901ea, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:06.735152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600f3dc-29e4-42e6-a97d-e4966e28fabf admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:06.748979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1054/2112] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:06 2022] PUT /identity/v3/projects/3d886cff441f4a54a659a1393fb77a40/users/89a6d355c3174f6ab99ae4d0cdb901ea/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:06.754824 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503292a-eb9a-4d7a-9bd8-2609b5c34c79 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:06.759769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc776d-c44e-487b-b016-82c8a2217799 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:06.760114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc776d-c44e-487b-b016-82c8a2217799 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:06.760445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc776d-c44e-487b-b016-82c8a2217799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:06.793966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-11bc776d-c44e-487b-b016-82c8a2217799 None None] MFA Rules not processed for user `89a6d355c3174f6ab99ae4d0cdb901ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:06.795178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-11bc776d-c44e-487b-b016-82c8a2217799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:06.820123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:06.820815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:06.821090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:06.821355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] PATH_INFO: `/v3/projects/6bbc4217ea2148108dc2165708dc9a48` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:06.822477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bbc4217ea2148108dc2165708dc9a48)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:06.824576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:06.869013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1055/2113] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:06.877012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc776d-c44e-487b-b016-82c8a2217799 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:06.877565 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bbc4217ea2148108dc2165708dc9a48', 'request_id': 'req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.878483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-11bc776d-c44e-487b-b016-82c8a2217799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:06.883835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bbc4217ea2148108dc2165708dc9a48', 'request_id': 'req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.889608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bbc4217ea2148108dc2165708dc9a48', 'request_id': 'req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.911576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] The token cache is being invalidate because project 6bbc4217ea2148108dc2165708dc9a48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:06.911861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.912994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:06.915052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1059/2114] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:06 2022] DELETE /identity/v3/projects/6bbc4217ea2148108dc2165708dc9a48 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:06.918800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:06.920564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:06.942610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc776d-c44e-487b-b016-82c8a2217799 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:06.951204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:06.995976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2106d7-8f7e-4998-8cd1-ef483e0c25f6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:07.002999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60be4f7-e17f-4ac9-b87c-2456a6329156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:07.003505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:07.003610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60be4f7-e17f-4ac9-b87c-2456a6329156 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.003878 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60be4f7-e17f-4ac9-b87c-2456a6329156 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.004126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.004239 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60be4f7-e17f-4ac9-b87c-2456a6329156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.004403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.004699 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] PATH_INFO: `/v3/users/73c7875f4aa4428f9d3c4faa4effdc5c` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.005831 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c7875f4aa4428f9d3c4faa4effdc5c)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:07.006146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e60be4f7-e17f-4ac9-b87c-2456a6329156 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:07.007747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:07.029415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:07.059305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c7875f4aa4428f9d3c4faa4effdc5c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.066573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c7875f4aa4428f9d3c4faa4effdc5c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.073999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60be4f7-e17f-4ac9-b87c-2456a6329156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:07.075439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60be4f7-e17f-4ac9-b87c-2456a6329156 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:07.078559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c7875f4aa4428f9d3c4faa4effdc5c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.086273 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1056/2115] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:07.087857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c7875f4aa4428f9d3c4faa4effdc5c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.088555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c7875f4aa4428f9d3c4faa4effdc5c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.095240 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1060/2116] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:06 2022] DELETE /identity/v3/users/73c7875f4aa4428f9d3c4faa4effdc5c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:07.388181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60be4f7-e17f-4ac9-b87c-2456a6329156 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:07.421477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60be4f7-e17f-4ac9-b87c-2456a6329156 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:07.487641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:07.488421 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.489184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.490142 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46 admin admin] PATH_INFO: `/v3/projects/896051e516ed49f6bfabcf3b0711b000` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.492548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=896051e516ed49f6bfabcf3b0711b000)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:07.494578 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:07.552761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896051e516ed49f6bfabcf3b0711b000', 'request_id': 'req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.562030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896051e516ed49f6bfabcf3b0711b000', 'request_id': 'req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.569518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:07.569984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896051e516ed49f6bfabcf3b0711b000', 'request_id': 'req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.571072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:07.573679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:07.583414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46 admin admin] The token cache is being invalidate because project 896051e516ed49f6bfabcf3b0711b000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:07.583902 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.584772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fd28fbcc-78e8-488f-8d9f-4ae9babcfa46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.585199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000019bc-1ac0-43e9-8010-946f3885bfd5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:07.586496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1057/2117] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:07 2022] DELETE /identity/v3/projects/896051e516ed49f6bfabcf3b0711b000 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:07.591789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8f70b-61be-4252-9a9b-56cbc84223e7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.592070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8f70b-61be-4252-9a9b-56cbc84223e7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.592341 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8f70b-61be-4252-9a9b-56cbc84223e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.613712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-8fb8f70b-61be-4252-9a9b-56cbc84223e7 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:07.613712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8fb8f70b-61be-4252-9a9b-56cbc84223e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:07.650922 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495968ce-28c8-45be-963c-e2d35a6c86ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:07.651551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495968ce-28c8-45be-963c-e2d35a6c86ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.651895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495968ce-28c8-45be-963c-e2d35a6c86ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.652233 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495968ce-28c8-45be-963c-e2d35a6c86ea admin admin] PATH_INFO: `/v3/users/1d08531c294f451f898abc384a365d01` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.653508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495968ce-28c8-45be-963c-e2d35a6c86ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d08531c294f451f898abc384a365d01)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:07.655523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495968ce-28c8-45be-963c-e2d35a6c86ea admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:07.681052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1058/2118] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:07.691180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5a5cf-c3e7-4d0b-9496-28b4f387d207 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.691459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5a5cf-c3e7-4d0b-9496-28b4f387d207 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.691737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5a5cf-c3e7-4d0b-9496-28b4f387d207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.709276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-495968ce-28c8-45be-963c-e2d35a6c86ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d08531c294f451f898abc384a365d01'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.711405 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2ca5a5cf-c3e7-4d0b-9496-28b4f387d207 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:07.712706 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2ca5a5cf-c3e7-4d0b-9496-28b4f387d207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:07.722652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1059/2119] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:07.723805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-495968ce-28c8-45be-963c-e2d35a6c86ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d08531c294f451f898abc384a365d01'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.727185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5a5cf-c3e7-4d0b-9496-28b4f387d207 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:07.736828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5a5cf-c3e7-4d0b-9496-28b4f387d207 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:07.737817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-495968ce-28c8-45be-963c-e2d35a6c86ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d08531c294f451f898abc384a365d01'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.741866 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bdb8b-a40b-4815-abb5-3c4c7eac3618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:07.742470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bdb8b-a40b-4815-abb5-3c4c7eac3618 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.742877 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bdb8b-a40b-4815-abb5-3c4c7eac3618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.743160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bdb8b-a40b-4815-abb5-3c4c7eac3618 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.743839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bdb8b-a40b-4815-abb5-3c4c7eac3618 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:07.744172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bdb8b-a40b-4815-abb5-3c4c7eac3618 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:07.745960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bdb8b-a40b-4815-abb5-3c4c7eac3618 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:07.747024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-495968ce-28c8-45be-963c-e2d35a6c86ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d08531c294f451f898abc384a365d01'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.747634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-495968ce-28c8-45be-963c-e2d35a6c86ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d08531c294f451f898abc384a365d01'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:07.753764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1061/2120] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:07 2022] DELETE /identity/v3/users/1d08531c294f451f898abc384a365d01 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:07.758233 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f4e78-addf-45a3-bb4e-6b3f2738dd27 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.758511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f4e78-addf-45a3-bb4e-6b3f2738dd27 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.758935 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f4e78-addf-45a3-bb4e-6b3f2738dd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.763529 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1060/2121] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:20:07.764707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bdb8b-a40b-4815-abb5-3c4c7eac3618 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:07.766496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9f1bdb8b-a40b-4815-abb5-3c4c7eac3618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:07.778503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-061f4e78-addf-45a3-bb4e-6b3f2738dd27 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:07.779713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-061f4e78-addf-45a3-bb4e-6b3f2738dd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:07.780488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bdb8b-a40b-4815-abb5-3c4c7eac3618 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:07.790279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:07.846912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccb7d2-0b5b-46d5-b70d-79948d20bb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:07.847629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccb7d2-0b5b-46d5-b70d-79948d20bb1c admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.847984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccb7d2-0b5b-46d5-b70d-79948d20bb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.848657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccb7d2-0b5b-46d5-b70d-79948d20bb1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.848770 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1062/2122] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:07.849147 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccb7d2-0b5b-46d5-b70d-79948d20bb1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:07.851238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccb7d2-0b5b-46d5-b70d-79948d20bb1c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:07.858392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f4e78-addf-45a3-bb4e-6b3f2738dd27 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:07.859743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-061f4e78-addf-45a3-bb4e-6b3f2738dd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:07.888811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1061/2123] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:07 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:20:07.890160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccb7d2-0b5b-46d5-b70d-79948d20bb1c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:07.924842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f4e78-addf-45a3-bb4e-6b3f2738dd27 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:07.958492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccb7d2-0b5b-46d5-b70d-79948d20bb1c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:07.963653 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68f52a-bc61-41d4-83d7-481bb5da6a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:07.964292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68f52a-bc61-41d4-83d7-481bb5da6a83 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.964567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68f52a-bc61-41d4-83d7-481bb5da6a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.964901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68f52a-bc61-41d4-83d7-481bb5da6a83 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.965533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68f52a-bc61-41d4-83d7-481bb5da6a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:07.967487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68f52a-bc61-41d4-83d7-481bb5da6a83 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:07.975955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:07.978494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f7872-02e3-489b-9ef7-8e0215c9bd5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:07.979027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f7872-02e3-489b-9ef7-8e0215c9bd5a service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:07.979311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f7872-02e3-489b-9ef7-8e0215c9bd5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:07.979645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f7872-02e3-489b-9ef7-8e0215c9bd5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:07.990046 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f7872-02e3-489b-9ef7-8e0215c9bd5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:07.991514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f7872-02e3-489b-9ef7-8e0215c9bd5a service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:07.995480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1062/2124] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:07 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:07.996496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68f52a-bc61-41d4-83d7-481bb5da6a83 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:08.003354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1063/2125] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:08.008179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68f52a-bc61-41d4-83d7-481bb5da6a83 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:08.014800 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bfd4d-a9be-4bc8-9b56-0ae385733813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:08.015415 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bfd4d-a9be-4bc8-9b56-0ae385733813 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:08.015683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bfd4d-a9be-4bc8-9b56-0ae385733813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:08.015944 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bfd4d-a9be-4bc8-9b56-0ae385733813 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:08.016886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bfd4d-a9be-4bc8-9b56-0ae385733813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:08.018864 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bfd4d-a9be-4bc8-9b56-0ae385733813 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:08.034795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1063/2126] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:08.038706 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f7872-02e3-489b-9ef7-8e0215c9bd5a service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:08.048771 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f7872-02e3-489b-9ef7-8e0215c9bd5a service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:08.056038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d9b3b-71d1-4c0e-9621-7fd177b4a0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:08.056676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d9b3b-71d1-4c0e-9621-7fd177b4a0ac admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:08.056941 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d9b3b-71d1-4c0e-9621-7fd177b4a0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:08.057213 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d9b3b-71d1-4c0e-9621-7fd177b4a0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:08.057864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d9b3b-71d1-4c0e-9621-7fd177b4a0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:08.059522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d9b3b-71d1-4c0e-9621-7fd177b4a0ac admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:08.070676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1064/2127] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:08.075072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bfd4d-a9be-4bc8-9b56-0ae385733813 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:08.085474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bfd4d-a9be-4bc8-9b56-0ae385733813 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:08.090540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd155392-f9e4-44a5-bd75-a2347b810e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:08.091196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd155392-f9e4-44a5-bd75-a2347b810e37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:08.091469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd155392-f9e4-44a5-bd75-a2347b810e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:08.091737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd155392-f9e4-44a5-bd75-a2347b810e37 admin admin] PATH_INFO: `/v3/projects/8aae73ba0d4f4649859607a7220d2f10/users/76565411b3d94941876e1168296781f7/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:08.108945 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd155392-f9e4-44a5-bd75-a2347b810e37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aae73ba0d4f4649859607a7220d2f10, user_id=76565411b3d94941876e1168296781f7, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:08.111872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd155392-f9e4-44a5-bd75-a2347b810e37 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:08.125394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1064/2128] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:08 2022] PUT /identity/v3/projects/8aae73ba0d4f4649859607a7220d2f10/users/76565411b3d94941876e1168296781f7/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:08.136022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb552e-4805-4215-9cf2-d45c1bf79378 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:08.136302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb552e-4805-4215-9cf2-d45c1bf79378 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:08.136569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb552e-4805-4215-9cf2-d45c1bf79378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:08.171730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-3bdb552e-4805-4215-9cf2-d45c1bf79378 None None] MFA Rules not processed for user `76565411b3d94941876e1168296781f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:08.173397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3bdb552e-4805-4215-9cf2-d45c1bf79378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:08.228763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:08.246660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1065/2129] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:08.254722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd155392-f9e4-44a5-bd75-a2347b810e37 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:08.256875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-dd155392-f9e4-44a5-bd75-a2347b810e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:08.289617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb552e-4805-4215-9cf2-d45c1bf79378 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:08.291144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3bdb552e-4805-4215-9cf2-d45c1bf79378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:08.326370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd155392-f9e4-44a5-bd75-a2347b810e37 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:08.330518 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896011a-daf7-49f6-99c7-db65b8debf13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:08.331123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896011a-daf7-49f6-99c7-db65b8debf13 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:08.331396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896011a-daf7-49f6-99c7-db65b8debf13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:08.331655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896011a-daf7-49f6-99c7-db65b8debf13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:08.341042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896011a-daf7-49f6-99c7-db65b8debf13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:08.342632 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896011a-daf7-49f6-99c7-db65b8debf13 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:08.351979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1065/2130] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:08.365094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb552e-4805-4215-9cf2-d45c1bf79378 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:08.369971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:08.370657 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:08.370984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:08.371280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] PATH_INFO: `/v3/projects/edd1c2e1025c4053b0d31e1d18482e44` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:08.379415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edd1c2e1025c4053b0d31e1d18482e44)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:08.379415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:08.380517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896011a-daf7-49f6-99c7-db65b8debf13 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:08.391659 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896011a-daf7-49f6-99c7-db65b8debf13 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:08.396371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73f65c-60c1-4e29-9429-afb680410990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:08.397024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73f65c-60c1-4e29-9429-afb680410990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:08.397300 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73f65c-60c1-4e29-9429-afb680410990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:08.397569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73f65c-60c1-4e29-9429-afb680410990 admin admin] PATH_INFO: `/v3/users/76565411b3d94941876e1168296781f7` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:08.398701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73f65c-60c1-4e29-9429-afb680410990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76565411b3d94941876e1168296781f7)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:08.400692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73f65c-60c1-4e29-9429-afb680410990 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:08.427631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edd1c2e1025c4053b0d31e1d18482e44', 'request_id': 'req-a1000744-8e80-4026-ac01-bb0439090107'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.435112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edd1c2e1025c4053b0d31e1d18482e44', 'request_id': 'req-a1000744-8e80-4026-ac01-bb0439090107'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.440731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edd1c2e1025c4053b0d31e1d18482e44', 'request_id': 'req-a1000744-8e80-4026-ac01-bb0439090107'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.450718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9f73f65c-60c1-4e29-9429-afb680410990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76565411b3d94941876e1168296781f7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.451491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9f73f65c-60c1-4e29-9429-afb680410990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76565411b3d94941876e1168296781f7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.454257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] The token cache is being invalidate because project edd1c2e1025c4053b0d31e1d18482e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:08.454542 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.455264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.456994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1066/2131] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:08 2022] DELETE /identity/v3/projects/edd1c2e1025c4053b0d31e1d18482e44 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:08.458733 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9f73f65c-60c1-4e29-9429-afb680410990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76565411b3d94941876e1168296781f7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.460803 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:08.462153 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:08.471275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9f73f65c-60c1-4e29-9429-afb680410990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76565411b3d94941876e1168296781f7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.472664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9f73f65c-60c1-4e29-9429-afb680410990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76565411b3d94941876e1168296781f7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.487926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1066/2132] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:08 2022] DELETE /identity/v3/users/76565411b3d94941876e1168296781f7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:08.492950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90748a7-225c-4001-b2e1-c693c6d5f912 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:08.493227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90748a7-225c-4001-b2e1-c693c6d5f912 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:08.493496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90748a7-225c-4001-b2e1-c693c6d5f912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:08.512989 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b90748a7-225c-4001-b2e1-c693c6d5f912 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:08.514194 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b90748a7-225c-4001-b2e1-c693c6d5f912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:08.538472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1000744-8e80-4026-ac01-bb0439090107 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:08.544403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:08.589682 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1067/2133] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:08.590112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:08.601725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90748a7-225c-4001-b2e1-c693c6d5f912 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:08.603205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b90748a7-225c-4001-b2e1-c693c6d5f912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:08.623073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:08.623721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:08.623996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:08.624262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] PATH_INFO: `/v3/users/89a6d355c3174f6ab99ae4d0cdb901ea` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:08.625550 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=89a6d355c3174f6ab99ae4d0cdb901ea)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:08.627701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:08.670338 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90748a7-225c-4001-b2e1-c693c6d5f912 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:08.674740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cca34-a1e1-4202-8d89-0abd5f2f89c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:08.675389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cca34-a1e1-4202-8d89-0abd5f2f89c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:08.675666 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cca34-a1e1-4202-8d89-0abd5f2f89c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:08.675937 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cca34-a1e1-4202-8d89-0abd5f2f89c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:08.686458 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89a6d355c3174f6ab99ae4d0cdb901ea'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.690672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0cca34-a1e1-4202-8d89-0abd5f2f89c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:08.692508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0cca34-a1e1-4202-8d89-0abd5f2f89c4 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:08.695582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89a6d355c3174f6ab99ae4d0cdb901ea'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.710351 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89a6d355c3174f6ab99ae4d0cdb901ea'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.713248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1068/2134] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:08.721683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89a6d355c3174f6ab99ae4d0cdb901ea'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.722560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89a6d355c3174f6ab99ae4d0cdb901ea'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:08.748173 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1067/2135] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:08 2022] DELETE /identity/v3/users/89a6d355c3174f6ab99ae4d0cdb901ea => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:09.085711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cca34-a1e1-4202-8d89-0abd5f2f89c4 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:09.087269 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ba0cca34-a1e1-4202-8d89-0abd5f2f89c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:09.100175 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cca34-a1e1-4202-8d89-0abd5f2f89c4 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:09.167116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc008cd-0656-46f8-a371-6a0aff946f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:09.167813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc008cd-0656-46f8-a371-6a0aff946f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.168108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc008cd-0656-46f8-a371-6a0aff946f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.168454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc008cd-0656-46f8-a371-6a0aff946f79 admin admin] PATH_INFO: `/v3/projects/8aae73ba0d4f4649859607a7220d2f10` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.169917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc008cd-0656-46f8-a371-6a0aff946f79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aae73ba0d4f4649859607a7220d2f10)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:09.171906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc008cd-0656-46f8-a371-6a0aff946f79 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:09.214537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:09.216322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:09.241547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c62a2e-e0e2-4970-9176-402c584e81ed admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:09.245197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-efc008cd-0656-46f8-a371-6a0aff946f79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aae73ba0d4f4649859607a7220d2f10', 'request_id': 'req-efc008cd-0656-46f8-a371-6a0aff946f79'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:09.255379 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-efc008cd-0656-46f8-a371-6a0aff946f79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aae73ba0d4f4649859607a7220d2f10', 'request_id': 'req-efc008cd-0656-46f8-a371-6a0aff946f79'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:09.263098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-efc008cd-0656-46f8-a371-6a0aff946f79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aae73ba0d4f4649859607a7220d2f10', 'request_id': 'req-efc008cd-0656-46f8-a371-6a0aff946f79'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:09.275473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-efc008cd-0656-46f8-a371-6a0aff946f79 admin admin] The token cache is being invalidate because project 8aae73ba0d4f4649859607a7220d2f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:09.275833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-efc008cd-0656-46f8-a371-6a0aff946f79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:09.277187 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-efc008cd-0656-46f8-a371-6a0aff946f79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:09.280682 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1069/2136] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:09 2022] DELETE /identity/v3/projects/8aae73ba0d4f4649859607a7220d2f10 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:09.284918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d850e4-fe75-4f5f-863e-d947437aa007 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.285282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d850e4-fe75-4f5f-863e-d947437aa007 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.285560 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d850e4-fe75-4f5f-863e-d947437aa007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.295482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:09.308982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f1d850e4-fe75-4f5f-863e-d947437aa007 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:09.310632 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f1d850e4-fe75-4f5f-863e-d947437aa007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:09.312392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462693b2-6d64-434a-96bd-1313ad070eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:09.313106 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462693b2-6d64-434a-96bd-1313ad070eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.313388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462693b2-6d64-434a-96bd-1313ad070eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.313653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462693b2-6d64-434a-96bd-1313ad070eb7 admin admin] PATH_INFO: `/v3/projects/3d886cff441f4a54a659a1393fb77a40` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.314993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462693b2-6d64-434a-96bd-1313ad070eb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d886cff441f4a54a659a1393fb77a40)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:09.317180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462693b2-6d64-434a-96bd-1313ad070eb7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:09.320432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1070/2137] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:09.328431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bea3d3-dd8f-4ff2-8e89-2da7e7a65982 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.329199 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bea3d3-dd8f-4ff2-8e89-2da7e7a65982 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.329662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bea3d3-dd8f-4ff2-8e89-2da7e7a65982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.352454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-79bea3d3-dd8f-4ff2-8e89-2da7e7a65982 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:09.353728 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-79bea3d3-dd8f-4ff2-8e89-2da7e7a65982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:09.371118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1071/2138] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:09.376615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bea3d3-dd8f-4ff2-8e89-2da7e7a65982 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:09.383868 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-462693b2-6d64-434a-96bd-1313ad070eb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d886cff441f4a54a659a1393fb77a40', 'request_id': 'req-462693b2-6d64-434a-96bd-1313ad070eb7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:09.385888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:09.387795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bea3d3-dd8f-4ff2-8e89-2da7e7a65982 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:09.391325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-462693b2-6d64-434a-96bd-1313ad070eb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d886cff441f4a54a659a1393fb77a40', 'request_id': 'req-462693b2-6d64-434a-96bd-1313ad070eb7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:09.399873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-462693b2-6d64-434a-96bd-1313ad070eb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d886cff441f4a54a659a1393fb77a40', 'request_id': 'req-462693b2-6d64-434a-96bd-1313ad070eb7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:09.416409 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-462693b2-6d64-434a-96bd-1313ad070eb7 admin admin] The token cache is being invalidate because project 3d886cff441f4a54a659a1393fb77a40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:09.416809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-462693b2-6d64-434a-96bd-1313ad070eb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:09.417487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-462693b2-6d64-434a-96bd-1313ad070eb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:09.419180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1068/2139] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:09 2022] DELETE /identity/v3/projects/3d886cff441f4a54a659a1393fb77a40 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:09.424219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77747ab0-465c-43c5-9069-3c65e833b82a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.424496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77747ab0-465c-43c5-9069-3c65e833b82a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.424804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77747ab0-465c-43c5-9069-3c65e833b82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.445591 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-77747ab0-465c-43c5-9069-3c65e833b82a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:09.446807 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-77747ab0-465c-43c5-9069-3c65e833b82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:09.458057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd23f2a-8b34-48f4-a29f-639e0701cd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:09.458709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd23f2a-8b34-48f4-a29f-639e0701cd5e admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.458981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd23f2a-8b34-48f4-a29f-639e0701cd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.459244 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd23f2a-8b34-48f4-a29f-639e0701cd5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.459902 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd23f2a-8b34-48f4-a29f-639e0701cd5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:09.460202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd23f2a-8b34-48f4-a29f-639e0701cd5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:09.461846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd23f2a-8b34-48f4-a29f-639e0701cd5e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:09.477938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1072/2140] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:20:09.478869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd23f2a-8b34-48f4-a29f-639e0701cd5e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:09.480863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5bd23f2a-8b34-48f4-a29f-639e0701cd5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:09.496842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd23f2a-8b34-48f4-a29f-639e0701cd5e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:09.502448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5369aa3-f3f8-45a7-bb7d-a754ccadd0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:09.503085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5369aa3-f3f8-45a7-bb7d-a754ccadd0ef admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.503524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5369aa3-f3f8-45a7-bb7d-a754ccadd0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.503795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5369aa3-f3f8-45a7-bb7d-a754ccadd0ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.504435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5369aa3-f3f8-45a7-bb7d-a754ccadd0ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:09.506486 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5369aa3-f3f8-45a7-bb7d-a754ccadd0ef admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:09.525858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1069/2141] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:09.534961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d485-d071-42de-9365-e1b3f91f2740 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.535270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d485-d071-42de-9365-e1b3f91f2740 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.535547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5d485-d071-42de-9365-e1b3f91f2740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.549823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1073/2142] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:09 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:09.557091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5369aa3-f3f8-45a7-bb7d-a754ccadd0ef admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:09.562187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5bc5d485-d071-42de-9365-e1b3f91f2740 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:09.563609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5bc5d485-d071-42de-9365-e1b3f91f2740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:09.631783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5369aa3-f3f8-45a7-bb7d-a754ccadd0ef admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:09.636568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6b1c1-0cb9-48e1-bab6-3826866110ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:09.637145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6b1c1-0cb9-48e1-bab6-3826866110ad admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.637419 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6b1c1-0cb9-48e1-bab6-3826866110ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.637685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6b1c1-0cb9-48e1-bab6-3826866110ad admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.638496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6b1c1-0cb9-48e1-bab6-3826866110ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:09.640421 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6b1c1-0cb9-48e1-bab6-3826866110ad admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:09.646156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1070/2143] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:09.652108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5d485-d071-42de-9365-e1b3f91f2740 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:09.663203 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc5d485-d071-42de-9365-e1b3f91f2740 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:09.668638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1074/2144] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:09 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:20:09.672131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65119e-e92f-4981-8e06-19feff7eef62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:09.672571 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6b1c1-0cb9-48e1-bab6-3826866110ad admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:09.673149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65119e-e92f-4981-8e06-19feff7eef62 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.673605 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65119e-e92f-4981-8e06-19feff7eef62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.674038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65119e-e92f-4981-8e06-19feff7eef62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.675078 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65119e-e92f-4981-8e06-19feff7eef62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:09.675610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65119e-e92f-4981-8e06-19feff7eef62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:09.678343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65119e-e92f-4981-8e06-19feff7eef62 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:09.685055 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6b1c1-0cb9-48e1-bab6-3826866110ad admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:09.691419 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fb2ab-6a85-4c56-9161-71377bdd07bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:09.694283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1071/2145] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:20:09.696376 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fb2ab-6a85-4c56-9161-71377bdd07bb admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.696662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fb2ab-6a85-4c56-9161-71377bdd07bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.698850 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fb2ab-6a85-4c56-9161-71377bdd07bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.699539 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fb2ab-6a85-4c56-9161-71377bdd07bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:09.699845 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65119e-e92f-4981-8e06-19feff7eef62 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:09.704610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fb2ab-6a85-4c56-9161-71377bdd07bb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:09.714466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65119e-e92f-4981-8e06-19feff7eef62 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:09.720148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1075/2146] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:09.722129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc7a8c-7124-4a99-b327-d12c0d7dddce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:09.722746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc7a8c-7124-4a99-b327-d12c0d7dddce admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.723200 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc7a8c-7124-4a99-b327-d12c0d7dddce admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.723630 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc7a8c-7124-4a99-b327-d12c0d7dddce admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.724328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc7a8c-7124-4a99-b327-d12c0d7dddce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:09.728835 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fb2ab-6a85-4c56-9161-71377bdd07bb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:09.728988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc7a8c-7124-4a99-b327-d12c0d7dddce admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:09.739562 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fb2ab-6a85-4c56-9161-71377bdd07bb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:09.747697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d129d8a-2ab8-4640-b723-2e603e3095ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:09.748235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d129d8a-2ab8-4640-b723-2e603e3095ef admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.748476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d129d8a-2ab8-4640-b723-2e603e3095ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.748775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d129d8a-2ab8-4640-b723-2e603e3095ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.749838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d129d8a-2ab8-4640-b723-2e603e3095ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:09.751100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d129d8a-2ab8-4640-b723-2e603e3095ef admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:09.769922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1076/2147] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:09.771582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1072/2148] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:09 2022] POST /identity/v3/projects => generated 355 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:09.774508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d129d8a-2ab8-4640-b723-2e603e3095ef admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:09.775933 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc7a8c-7124-4a99-b327-d12c0d7dddce admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:09.792534 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d129d8a-2ab8-4640-b723-2e603e3095ef admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:09.844212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:09.857567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:09.883392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc7a8c-7124-4a99-b327-d12c0d7dddce admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:09.883941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0802e82-268b-44c4-b0a7-be2b7d5f2f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:09.884754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0802e82-268b-44c4-b0a7-be2b7d5f2f9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.885120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0802e82-268b-44c4-b0a7-be2b7d5f2f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.886473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0802e82-268b-44c4-b0a7-be2b7d5f2f9f admin admin] PATH_INFO: `/v3/projects/592a763d7123405fb0576b00edfcd1bf/users/3077d51be16e40d9bb70b984583c0f8a/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.889495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a8e8d-3fc2-4b56-9d92-54aac9a91274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:09.889638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a8e8d-3fc2-4b56-9d92-54aac9a91274 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.889897 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a8e8d-3fc2-4b56-9d92-54aac9a91274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.890163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a8e8d-3fc2-4b56-9d92-54aac9a91274 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.890811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a8e8d-3fc2-4b56-9d92-54aac9a91274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:09.895044 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a8e8d-3fc2-4b56-9d92-54aac9a91274 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:09.911068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0802e82-268b-44c4-b0a7-be2b7d5f2f9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=592a763d7123405fb0576b00edfcd1bf, user_id=3077d51be16e40d9bb70b984583c0f8a, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:09.914156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0802e82-268b-44c4-b0a7-be2b7d5f2f9f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:09.923702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1073/2149] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:09 2022] POST /identity/v3/users => generated 278 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:20:09.928183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a8e8d-3fc2-4b56-9d92-54aac9a91274 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:09.932541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1077/2150] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:09 2022] PUT /identity/v3/projects/592a763d7123405fb0576b00edfcd1bf/users/3077d51be16e40d9bb70b984583c0f8a/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:09.937979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cd970-6f6b-4784-bf34-9a80b32e33b2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:09.938412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cd970-6f6b-4784-bf34-9a80b32e33b2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:09.938892 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cd970-6f6b-4784-bf34-9a80b32e33b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:09.949119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a8e8d-3fc2-4b56-9d92-54aac9a91274 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:09.966010 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c36cd970-6f6b-4784-bf34-9a80b32e33b2 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:09.967347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c36cd970-6f6b-4784-bf34-9a80b32e33b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:10.015684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf365d9-5c6c-4089-aa13-a5019f39aacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:10.017244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf365d9-5c6c-4089-aa13-a5019f39aacb admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:10.017767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf365d9-5c6c-4089-aa13-a5019f39aacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:10.018117 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf365d9-5c6c-4089-aa13-a5019f39aacb admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.018833 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf365d9-5c6c-4089-aa13-a5019f39aacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:10.021772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf365d9-5c6c-4089-aa13-a5019f39aacb admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:10.035844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1078/2151] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:10.036792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1074/2152] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:20:10.040168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cd970-6f6b-4784-bf34-9a80b32e33b2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:10.046509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf365d9-5c6c-4089-aa13-a5019f39aacb admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:10.048451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-acf365d9-5c6c-4089-aa13-a5019f39aacb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:10.052982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cd970-6f6b-4784-bf34-9a80b32e33b2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:10.058993 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b1e7e-d890-448f-a052-86b7705334b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:10.059614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b1e7e-d890-448f-a052-86b7705334b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:10.060216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b1e7e-d890-448f-a052-86b7705334b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:10.060216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b1e7e-d890-448f-a052-86b7705334b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.060833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b1e7e-d890-448f-a052-86b7705334b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:10.062483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b1e7e-d890-448f-a052-86b7705334b5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:10.077102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1079/2153] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:10.081573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b1e7e-d890-448f-a052-86b7705334b5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:10.099488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b1e7e-d890-448f-a052-86b7705334b5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:10.105155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562ab75-3945-468c-b178-b7c25f3fc6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:10.105856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562ab75-3945-468c-b178-b7c25f3fc6c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:10.106128 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562ab75-3945-468c-b178-b7c25f3fc6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:10.106395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562ab75-3945-468c-b178-b7c25f3fc6c8 admin admin] PATH_INFO: `/v3/projects/404c226fd2d9486b804f6b2670e4a824/users/2145b870f54f4c71bd02c0153cb9bab6/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.122615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562ab75-3945-468c-b178-b7c25f3fc6c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=404c226fd2d9486b804f6b2670e4a824, user_id=2145b870f54f4c71bd02c0153cb9bab6, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:10.125343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562ab75-3945-468c-b178-b7c25f3fc6c8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:10.126892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf365d9-5c6c-4089-aa13-a5019f39aacb admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:10.131663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630cad9-0f00-4933-9d35-a17c7ef22014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:10.132288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630cad9-0f00-4933-9d35-a17c7ef22014 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:10.132560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630cad9-0f00-4933-9d35-a17c7ef22014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:10.132900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630cad9-0f00-4933-9d35-a17c7ef22014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.141065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1080/2154] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:10 2022] PUT /identity/v3/projects/404c226fd2d9486b804f6b2670e4a824/users/2145b870f54f4c71bd02c0153cb9bab6/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:10.143627 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630cad9-0f00-4933-9d35-a17c7ef22014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:10.145118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630cad9-0f00-4933-9d35-a17c7ef22014 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:10.154939 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4e1db-04dc-4447-98d7-d4650809cf95 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:10.155223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4e1db-04dc-4447-98d7-d4650809cf95 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:10.155490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4e1db-04dc-4447-98d7-d4650809cf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.155619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1075/2155] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:10.186491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d0d4e1db-04dc-4447-98d7-d4650809cf95 None None] MFA Rules not processed for user `2145b870f54f4c71bd02c0153cb9bab6`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:10.187783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d0d4e1db-04dc-4447-98d7-d4650809cf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:10.269328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1081/2156] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:10.274687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630cad9-0f00-4933-9d35-a17c7ef22014 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:10.286764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630cad9-0f00-4933-9d35-a17c7ef22014 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:10.370736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30229d0-2d5c-4a11-8dd8-c40c09e915ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:10.373034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30229d0-2d5c-4a11-8dd8-c40c09e915ef admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:10.374619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30229d0-2d5c-4a11-8dd8-c40c09e915ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:10.374823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30229d0-2d5c-4a11-8dd8-c40c09e915ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.375647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30229d0-2d5c-4a11-8dd8-c40c09e915ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:10.378136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30229d0-2d5c-4a11-8dd8-c40c09e915ef admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:10.404457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:10.416651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1076/2157] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:10 2022] POST /identity/v3/projects => generated 355 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:20:10.424790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4e1db-04dc-4447-98d7-d4650809cf95 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:10.436812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4e1db-04dc-4447-98d7-d4650809cf95 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:10.440684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:10.499017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83049c57-3b11-41d8-8e84-3af041583463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:10.499712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83049c57-3b11-41d8-8e84-3af041583463 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:10.499966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83049c57-3b11-41d8-8e84-3af041583463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:10.500257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83049c57-3b11-41d8-8e84-3af041583463 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.501085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83049c57-3b11-41d8-8e84-3af041583463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:10.503163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83049c57-3b11-41d8-8e84-3af041583463 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:10.530517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1082/2158] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:10 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:10.534824 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30229d0-2d5c-4a11-8dd8-c40c09e915ef admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:10.550936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30229d0-2d5c-4a11-8dd8-c40c09e915ef admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:10.556074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb3134-4dd6-4732-8a57-76697cb974bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:10.556808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb3134-4dd6-4732-8a57-76697cb974bd admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:10.557137 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb3134-4dd6-4732-8a57-76697cb974bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:10.557490 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb3134-4dd6-4732-8a57-76697cb974bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.558218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb3134-4dd6-4732-8a57-76697cb974bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:10.559885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb3134-4dd6-4732-8a57-76697cb974bd admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:10.571675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1077/2159] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:10.576281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83049c57-3b11-41d8-8e84-3af041583463 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:10.587736 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83049c57-3b11-41d8-8e84-3af041583463 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:10.593021 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be66a8-6570-400b-9964-8fe88572a9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:10.593694 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be66a8-6570-400b-9964-8fe88572a9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:10.593948 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be66a8-6570-400b-9964-8fe88572a9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:10.594223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be66a8-6570-400b-9964-8fe88572a9bf admin admin] PATH_INFO: `/v3/projects/a85a3fc1f44b41cb8ded83154b3c12f5/users/a05b778a57e54aeb840c33f623c35f95/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.611319 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be66a8-6570-400b-9964-8fe88572a9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85a3fc1f44b41cb8ded83154b3c12f5, user_id=a05b778a57e54aeb840c33f623c35f95, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:10.614320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be66a8-6570-400b-9964-8fe88572a9bf admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:10.632781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1083/2160] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:10 2022] PUT /identity/v3/projects/a85a3fc1f44b41cb8ded83154b3c12f5/users/a05b778a57e54aeb840c33f623c35f95/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:10.637214 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb3134-4dd6-4732-8a57-76697cb974bd admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:10.648915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb3134-4dd6-4732-8a57-76697cb974bd admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:10.718668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb88a3-f073-49c9-9a84-ce1fe74ffebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:10.719472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb88a3-f073-49c9-9a84-ce1fe74ffebe admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:10.719781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb88a3-f073-49c9-9a84-ce1fe74ffebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:10.720234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb88a3-f073-49c9-9a84-ce1fe74ffebe admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.720936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb88a3-f073-49c9-9a84-ce1fe74ffebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:10.722654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb88a3-f073-49c9-9a84-ce1fe74ffebe admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:10.733595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1078/2161] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:10.738072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be66a8-6570-400b-9964-8fe88572a9bf admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:10.749735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be66a8-6570-400b-9964-8fe88572a9bf admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:10.755143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ef73c-e7d2-4145-a244-2d2b9e176684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:10.755814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ef73c-e7d2-4145-a244-2d2b9e176684 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:10.756090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ef73c-e7d2-4145-a244-2d2b9e176684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:10.756365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ef73c-e7d2-4145-a244-2d2b9e176684 admin admin] PATH_INFO: `/v3/projects/a85a3fc1f44b41cb8ded83154b3c12f5/users/a05b778a57e54aeb840c33f623c35f95/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.758573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ef73c-e7d2-4145-a244-2d2b9e176684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85a3fc1f44b41cb8ded83154b3c12f5, user_id=a05b778a57e54aeb840c33f623c35f95, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:10.761268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ef73c-e7d2-4145-a244-2d2b9e176684 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:10.773215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1084/2162] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:10 2022] PUT /identity/v3/projects/a85a3fc1f44b41cb8ded83154b3c12f5/users/a05b778a57e54aeb840c33f623c35f95/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:10.784008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e599f-76e5-4592-ba30-49fd569f7768 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:10.784326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e599f-76e5-4592-ba30-49fd569f7768 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:10.784679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e599f-76e5-4592-ba30-49fd569f7768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.814225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-652e599f-76e5-4592-ba30-49fd569f7768 None None] MFA Rules not processed for user `a05b778a57e54aeb840c33f623c35f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:10.815548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-652e599f-76e5-4592-ba30-49fd569f7768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:10.902600 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1079/2163] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:10 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:10.915118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ef73c-e7d2-4145-a244-2d2b9e176684 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:10.917586 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f43ef73c-e7d2-4145-a244-2d2b9e176684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:10.979154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ef73c-e7d2-4145-a244-2d2b9e176684 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:10.982897 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0567ac9-495a-4110-8b1d-f26b0af11ce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:10.983514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0567ac9-495a-4110-8b1d-f26b0af11ce1 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:10.983789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0567ac9-495a-4110-8b1d-f26b0af11ce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:10.984055 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0567ac9-495a-4110-8b1d-f26b0af11ce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:10.994170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0567ac9-495a-4110-8b1d-f26b0af11ce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:10.995632 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0567ac9-495a-4110-8b1d-f26b0af11ce1 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:11.008378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1085/2164] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:10 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:11.111935 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e599f-76e5-4592-ba30-49fd569f7768 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:11.126750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e599f-76e5-4592-ba30-49fd569f7768 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:11.135652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4ba1e-c646-47cf-9310-aba09bbb3927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:11.136353 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4ba1e-c646-47cf-9310-aba09bbb3927 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:11.137009 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4ba1e-c646-47cf-9310-aba09bbb3927 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:11.137422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4ba1e-c646-47cf-9310-aba09bbb3927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:11.222117 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4ba1e-c646-47cf-9310-aba09bbb3927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:11.223563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4ba1e-c646-47cf-9310-aba09bbb3927 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:11.240164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1080/2165] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:15.064632 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e3acc-b3fd-4fa3-b61d-5a732b9769af None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:15.065214 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e3acc-b3fd-4fa3-b61d-5a732b9769af None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:15.065214 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e3acc-b3fd-4fa3-b61d-5a732b9769af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:15.179741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-de3e3acc-b3fd-4fa3-b61d-5a732b9769af None None] MFA Rules not processed for user `3077d51be16e40d9bb70b984583c0f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:15.181655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-de3e3acc-b3fd-4fa3-b61d-5a732b9769af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:15.280445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:15.374930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1086/2166] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:15.377592 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4ba1e-c646-47cf-9310-aba09bbb3927 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:15.393754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:15.593862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4ba1e-c646-47cf-9310-aba09bbb3927 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:15.605484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44195c4-5698-4caa-b895-9ca9eb3f2849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:15.606058 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44195c4-5698-4caa-b895-9ca9eb3f2849 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:15.606335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44195c4-5698-4caa-b895-9ca9eb3f2849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:15.606791 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44195c4-5698-4caa-b895-9ca9eb3f2849 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:15.607423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44195c4-5698-4caa-b895-9ca9eb3f2849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:15.609534 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44195c4-5698-4caa-b895-9ca9eb3f2849 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:15.675081 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1081/2167] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:15 2022] POST /identity/v3/projects => generated 369 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:15.678903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e3acc-b3fd-4fa3-b61d-5a732b9769af None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:15.697342 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e3acc-b3fd-4fa3-b61d-5a732b9769af None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:15.843784 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bc7be-e435-4252-a266-f1c7cea15f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:15.844465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bc7be-e435-4252-a266-f1c7cea15f31 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:15.844804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bc7be-e435-4252-a266-f1c7cea15f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:15.845198 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bc7be-e435-4252-a266-f1c7cea15f31 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:15.845774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bc7be-e435-4252-a266-f1c7cea15f31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:15.847791 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bc7be-e435-4252-a266-f1c7cea15f31 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:15.897669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1087/2168] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:15 2022] POST /identity/v3/users => generated 291 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:15.901859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44195c4-5698-4caa-b895-9ca9eb3f2849 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:15.920391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44195c4-5698-4caa-b895-9ca9eb3f2849 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:15.954457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de44d41-4f38-4c6b-843e-94c21c50d3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:15.961104 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de44d41-4f38-4c6b-843e-94c21c50d3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:15.961369 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de44d41-4f38-4c6b-843e-94c21c50d3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:15.961656 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de44d41-4f38-4c6b-843e-94c21c50d3a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:15.962313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de44d41-4f38-4c6b-843e-94c21c50d3a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:15.963962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de44d41-4f38-4c6b-843e-94c21c50d3a8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:15.996697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1082/2169] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:16.000494 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bc7be-e435-4252-a266-f1c7cea15f31 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:16.017291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bc7be-e435-4252-a266-f1c7cea15f31 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:16.025496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab7be3-999f-42ee-a387-14f577fc313d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:16.027689 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab7be3-999f-42ee-a387-14f577fc313d admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:16.028709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab7be3-999f-42ee-a387-14f577fc313d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:16.031590 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab7be3-999f-42ee-a387-14f577fc313d admin admin] PATH_INFO: `/v3/projects/d965e9d40a02427aba973b31153c16a6/users/7081e82606914f3b975cdccaddf2088f/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:16.056172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab7be3-999f-42ee-a387-14f577fc313d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d965e9d40a02427aba973b31153c16a6, user_id=7081e82606914f3b975cdccaddf2088f, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:16.058693 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab7be3-999f-42ee-a387-14f577fc313d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:16.084647 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1088/2170] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:15 2022] PUT /identity/v3/projects/d965e9d40a02427aba973b31153c16a6/users/7081e82606914f3b975cdccaddf2088f/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:16.087366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de44d41-4f38-4c6b-843e-94c21c50d3a8 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:16.101574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de44d41-4f38-4c6b-843e-94c21c50d3a8 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:16.227036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6227a-1574-4544-9082-fe3e15adc9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:16.227781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6227a-1574-4544-9082-fe3e15adc9fc admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:16.228434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6227a-1574-4544-9082-fe3e15adc9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:16.228839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6227a-1574-4544-9082-fe3e15adc9fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:16.230195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6227a-1574-4544-9082-fe3e15adc9fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:16.233736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6227a-1574-4544-9082-fe3e15adc9fc admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:16.252024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1083/2171] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:16.253673 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab7be3-999f-42ee-a387-14f577fc313d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:16.268057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab7be3-999f-42ee-a387-14f577fc313d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:16.279333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0889276-9ef7-4cb3-a630-c61161ee2177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:16.280013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0889276-9ef7-4cb3-a630-c61161ee2177 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:16.280283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0889276-9ef7-4cb3-a630-c61161ee2177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:16.280552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0889276-9ef7-4cb3-a630-c61161ee2177 admin admin] PATH_INFO: `/v3/projects/d965e9d40a02427aba973b31153c16a6/users/7081e82606914f3b975cdccaddf2088f/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:16.287669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0889276-9ef7-4cb3-a630-c61161ee2177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d965e9d40a02427aba973b31153c16a6, user_id=7081e82606914f3b975cdccaddf2088f, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:16.291813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0889276-9ef7-4cb3-a630-c61161ee2177 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:16.311101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1089/2172] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:16 2022] PUT /identity/v3/projects/d965e9d40a02427aba973b31153c16a6/users/7081e82606914f3b975cdccaddf2088f/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:18.737233 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6227a-1574-4544-9082-fe3e15adc9fc admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:18.740200 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4ba6227a-1574-4544-9082-fe3e15adc9fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:18.763232 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6227a-1574-4544-9082-fe3e15adc9fc admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:18.857191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:18.858844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:18.859629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:18.860313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] PATH_INFO: `/v3/users/0df1bd200e744439b2363168f8687af5` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:18.862147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0df1bd200e744439b2363168f8687af5)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:18.864937 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:18.905062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:18.967171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0df1bd200e744439b2363168f8687af5'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:18.978471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0df1bd200e744439b2363168f8687af5'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:19.003511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0df1bd200e744439b2363168f8687af5'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:19.021281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0df1bd200e744439b2363168f8687af5'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:19.022056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0df1bd200e744439b2363168f8687af5'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:19.029300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1084/2173] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:18 2022] DELETE /identity/v3/users/0df1bd200e744439b2363168f8687af5 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:19.034868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb4fd6-7bab-4c09-a71a-605235908d9e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:19.035308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb4fd6-7bab-4c09-a71a-605235908d9e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:19.035308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb4fd6-7bab-4c09-a71a-605235908d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:19.083745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2ceb4fd6-7bab-4c09-a71a-605235908d9e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:19.087589 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2ceb4fd6-7bab-4c09-a71a-605235908d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:19.209114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1090/2174] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:19.238030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:19.257069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:19.412432 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b10070-1207-45ce-91f7-2b2cc7dd445e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:19.421871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f35a39-a069-4972-a7fa-437f3a5571af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:19.422559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f35a39-a069-4972-a7fa-437f3a5571af service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:19.422821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f35a39-a069-4972-a7fa-437f3a5571af service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:19.423080 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f35a39-a069-4972-a7fa-437f3a5571af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:19.432925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f35a39-a069-4972-a7fa-437f3a5571af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:19.434351 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f35a39-a069-4972-a7fa-437f3a5571af service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:19.445336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1085/2175] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:19.816170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb4fd6-7bab-4c09-a71a-605235908d9e None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:19.817953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2ceb4fd6-7bab-4c09-a71a-605235908d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:19.832558 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb4fd6-7bab-4c09-a71a-605235908d9e None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:19.843716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:19.844543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:19.844720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:19.844996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] PATH_INFO: `/v3/projects/1ecaa6d3fba4403c85276ec63d80dd34` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:19.846536 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ecaa6d3fba4403c85276ec63d80dd34)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:19.849609 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:19.951900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:19.973635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ecaa6d3fba4403c85276ec63d80dd34', 'request_id': 'req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:19.991009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ecaa6d3fba4403c85276ec63d80dd34', 'request_id': 'req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.015745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ecaa6d3fba4403c85276ec63d80dd34', 'request_id': 'req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.044300 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] The token cache is being invalidate because project 1ecaa6d3fba4403c85276ec63d80dd34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:20.044900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.045498 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.047298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1091/2176] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:19 2022] DELETE /identity/v3/projects/1ecaa6d3fba4403c85276ec63d80dd34 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:20.052206 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f35a39-a069-4972-a7fa-437f3a5571af service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:20.066540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f35a39-a069-4972-a7fa-437f3a5571af service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:20.208543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10187f-e546-4555-a8af-0c206ad2620c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:20.209254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10187f-e546-4555-a8af-0c206ad2620c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:20.209527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10187f-e546-4555-a8af-0c206ad2620c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:20.209792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10187f-e546-4555-a8af-0c206ad2620c admin admin] PATH_INFO: `/v3/users/290df346f2614738b980ed1d26159a70` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:20.211663 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10187f-e546-4555-a8af-0c206ad2620c admin admin] RBAC: Authorizing `identity:delete_user(user_id=290df346f2614738b980ed1d26159a70)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:20.214397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10187f-e546-4555-a8af-0c206ad2620c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:20.350523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:20.357518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8c10187f-e546-4555-a8af-0c206ad2620c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '290df346f2614738b980ed1d26159a70'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.390272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8c10187f-e546-4555-a8af-0c206ad2620c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '290df346f2614738b980ed1d26159a70'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.442027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8c10187f-e546-4555-a8af-0c206ad2620c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '290df346f2614738b980ed1d26159a70'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.460616 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8c10187f-e546-4555-a8af-0c206ad2620c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '290df346f2614738b980ed1d26159a70'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.461433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8c10187f-e546-4555-a8af-0c206ad2620c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '290df346f2614738b980ed1d26159a70'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:20.469459 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1086/2177] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:20 2022] DELETE /identity/v3/users/290df346f2614738b980ed1d26159a70 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:21.281487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:21.283261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:21.307629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95d5d8-d5a4-4abe-b659-f2d5082e2ff9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:21.424310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09a204-f940-43f4-9ede-451b4cd19b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:21.424983 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09a204-f940-43f4-9ede-451b4cd19b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.425267 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09a204-f940-43f4-9ede-451b4cd19b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.425552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09a204-f940-43f4-9ede-451b4cd19b8b admin admin] PATH_INFO: `/v3/projects/11d9008edaad44db8ad515f79e479554` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.427636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09a204-f940-43f4-9ede-451b4cd19b8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d9008edaad44db8ad515f79e479554)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:21.430068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09a204-f940-43f4-9ede-451b4cd19b8b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:21.491321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-da09a204-f940-43f4-9ede-451b4cd19b8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d9008edaad44db8ad515f79e479554', 'request_id': 'req-da09a204-f940-43f4-9ede-451b4cd19b8b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:21.502248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-da09a204-f940-43f4-9ede-451b4cd19b8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d9008edaad44db8ad515f79e479554', 'request_id': 'req-da09a204-f940-43f4-9ede-451b4cd19b8b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:21.509712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-da09a204-f940-43f4-9ede-451b4cd19b8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d9008edaad44db8ad515f79e479554', 'request_id': 'req-da09a204-f940-43f4-9ede-451b4cd19b8b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:21.525753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-da09a204-f940-43f4-9ede-451b4cd19b8b admin admin] The token cache is being invalidate because project 11d9008edaad44db8ad515f79e479554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:21.527046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-da09a204-f940-43f4-9ede-451b4cd19b8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:21.528310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-da09a204-f940-43f4-9ede-451b4cd19b8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:21.530311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1092/2178] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:21 2022] DELETE /identity/v3/projects/11d9008edaad44db8ad515f79e479554 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:21.536231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031ed03-829d-42ff-a110-2437a4b0b95a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.536733 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031ed03-829d-42ff-a110-2437a4b0b95a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.536835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031ed03-829d-42ff-a110-2437a4b0b95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.569445 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b031ed03-829d-42ff-a110-2437a4b0b95a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:21.570867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b031ed03-829d-42ff-a110-2437a4b0b95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:21.716096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1087/2179] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:21.723464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944a42e-929d-4fd5-afd2-26d6b5acc6b3 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.723737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944a42e-929d-4fd5-afd2-26d6b5acc6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.724007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944a42e-929d-4fd5-afd2-26d6b5acc6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.758953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3944a42e-929d-4fd5-afd2-26d6b5acc6b3 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:21.760200 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3944a42e-929d-4fd5-afd2-26d6b5acc6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:21.780139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1093/2180] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:21.785373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031ed03-829d-42ff-a110-2437a4b0b95a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:21.814476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031ed03-829d-42ff-a110-2437a4b0b95a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:21.839183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366697da-079c-4578-b081-952856f5e289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:21.841325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366697da-079c-4578-b081-952856f5e289 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.841325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366697da-079c-4578-b081-952856f5e289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.841325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366697da-079c-4578-b081-952856f5e289 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.841325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366697da-079c-4578-b081-952856f5e289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:21.841723 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366697da-079c-4578-b081-952856f5e289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:21.844748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366697da-079c-4578-b081-952856f5e289 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:21.866692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1088/2181] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:20:21.868010 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944a42e-929d-4fd5-afd2-26d6b5acc6b3 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:21.879459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944a42e-929d-4fd5-afd2-26d6b5acc6b3 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:21.899328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71069c-71b9-409f-ac36-186963c8cc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:21.900268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71069c-71b9-409f-ac36-186963c8cc08 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:21.901297 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71069c-71b9-409f-ac36-186963c8cc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:21.901964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71069c-71b9-409f-ac36-186963c8cc08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:21.904920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71069c-71b9-409f-ac36-186963c8cc08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:21.907294 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71069c-71b9-409f-ac36-186963c8cc08 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:21.981179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1094/2182] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:21 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:21.986651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366697da-079c-4578-b081-952856f5e289 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:21.998727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366697da-079c-4578-b081-952856f5e289 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.127490 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e30928-c2a8-46f1-86c3-98c945f8c3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.128776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e30928-c2a8-46f1-86c3-98c945f8c3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.129239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e30928-c2a8-46f1-86c3-98c945f8c3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.129650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e30928-c2a8-46f1-86c3-98c945f8c3d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.130502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e30928-c2a8-46f1-86c3-98c945f8c3d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.133659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e30928-c2a8-46f1-86c3-98c945f8c3d1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:22.194861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1089/2183] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:21 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:22.196218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71069c-71b9-409f-ac36-186963c8cc08 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:22.219434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71069c-71b9-409f-ac36-186963c8cc08 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.232476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e6cc6-ab82-42e5-8aeb-5509b8b2e16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.235556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e6cc6-ab82-42e5-8aeb-5509b8b2e16e admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.236567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e6cc6-ab82-42e5-8aeb-5509b8b2e16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.237430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e6cc6-ab82-42e5-8aeb-5509b8b2e16e admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.238879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e6cc6-ab82-42e5-8aeb-5509b8b2e16e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.244952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e6cc6-ab82-42e5-8aeb-5509b8b2e16e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:22.264673 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1095/2184] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:22.269280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e30928-c2a8-46f1-86c3-98c945f8c3d1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:22.286593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e30928-c2a8-46f1-86c3-98c945f8c3d1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.293469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2602302a-ff3b-4737-b10c-3cdae1e4281b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.294649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602302a-ff3b-4737-b10c-3cdae1e4281b admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.295067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602302a-ff3b-4737-b10c-3cdae1e4281b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.295475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602302a-ff3b-4737-b10c-3cdae1e4281b admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.296414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602302a-ff3b-4737-b10c-3cdae1e4281b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.298720 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2602302a-ff3b-4737-b10c-3cdae1e4281b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:22.312739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1090/2185] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:20:22.316846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e6cc6-ab82-42e5-8aeb-5509b8b2e16e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:22.330011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e6cc6-ab82-42e5-8aeb-5509b8b2e16e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.347451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11710467-4ef9-4a72-b037-de9487741b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.348391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11710467-4ef9-4a72-b037-de9487741b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.348870 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11710467-4ef9-4a72-b037-de9487741b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.349281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11710467-4ef9-4a72-b037-de9487741b2a admin admin] PATH_INFO: `/v3/projects/d488e48d8712453a89b4118b6825fcf2/users/282ee7dd9456437dbaf9e1b769cef810/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.387853 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11710467-4ef9-4a72-b037-de9487741b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d488e48d8712453a89b4118b6825fcf2, user_id=282ee7dd9456437dbaf9e1b769cef810, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.391193 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11710467-4ef9-4a72-b037-de9487741b2a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:22.394501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:22.426984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1096/2186] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:22 2022] PUT /identity/v3/projects/d488e48d8712453a89b4118b6825fcf2/users/282ee7dd9456437dbaf9e1b769cef810/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:22.438036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddc50f-537b-42f6-826e-6615aca9ce7d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.438338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddc50f-537b-42f6-826e-6615aca9ce7d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.438494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ddc50f-537b-42f6-826e-6615aca9ce7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.497840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-20ddc50f-537b-42f6-826e-6615aca9ce7d None None] MFA Rules not processed for user `282ee7dd9456437dbaf9e1b769cef810`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:22.499924 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-20ddc50f-537b-42f6-826e-6615aca9ce7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:22.517391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:22.619888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1091/2187] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:22.632512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11710467-4ef9-4a72-b037-de9487741b2a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:22.635465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-11710467-4ef9-4a72-b037-de9487741b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:22.769171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11710467-4ef9-4a72-b037-de9487741b2a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.776114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f4528-9ce8-4631-8e68-9c94056687a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.776975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f4528-9ce8-4631-8e68-9c94056687a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.777284 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f4528-9ce8-4631-8e68-9c94056687a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.777661 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f4528-9ce8-4631-8e68-9c94056687a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.788188 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f4528-9ce8-4631-8e68-9c94056687a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.789681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f4528-9ce8-4631-8e68-9c94056687a5 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:22.803144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1097/2188] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:22.849266 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddc50f-537b-42f6-826e-6615aca9ce7d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:22.874160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40f63b-58c3-4e8c-a589-0b90511a3e05 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.874941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40f63b-58c3-4e8c-a589-0b90511a3e05 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.875360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40f63b-58c3-4e8c-a589-0b90511a3e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.916137 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-bf40f63b-58c3-4e8c-a589-0b90511a3e05 None None] MFA Rules not processed for user `7081e82606914f3b975cdccaddf2088f`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:22.917443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bf40f63b-58c3-4e8c-a589-0b90511a3e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:22.950204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ddc50f-537b-42f6-826e-6615aca9ce7d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:22.956923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b123cf-0ad3-4de0-b3ca-4591360b4190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:22.957639 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b123cf-0ad3-4de0-b3ca-4591360b4190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:22.957916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b123cf-0ad3-4de0-b3ca-4591360b4190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:22.958183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b123cf-0ad3-4de0-b3ca-4591360b4190 admin admin] PATH_INFO: `/v3/users/282ee7dd9456437dbaf9e1b769cef810` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:22.959376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b123cf-0ad3-4de0-b3ca-4591360b4190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=282ee7dd9456437dbaf9e1b769cef810)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:22.961297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b123cf-0ad3-4de0-b3ca-4591360b4190 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:23.006666 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1098/2189] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:23.021258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40f63b-58c3-4e8c-a589-0b90511a3e05 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:23.022434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bf40f63b-58c3-4e8c-a589-0b90511a3e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:23.028815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-41b123cf-0ad3-4de0-b3ca-4591360b4190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '282ee7dd9456437dbaf9e1b769cef810'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:23.047693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-41b123cf-0ad3-4de0-b3ca-4591360b4190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '282ee7dd9456437dbaf9e1b769cef810'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:23.060263 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-41b123cf-0ad3-4de0-b3ca-4591360b4190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '282ee7dd9456437dbaf9e1b769cef810'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:23.071984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-41b123cf-0ad3-4de0-b3ca-4591360b4190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '282ee7dd9456437dbaf9e1b769cef810'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:23.072834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-41b123cf-0ad3-4de0-b3ca-4591360b4190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '282ee7dd9456437dbaf9e1b769cef810'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:23.081061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1092/2190] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:22 2022] DELETE /identity/v3/users/282ee7dd9456437dbaf9e1b769cef810 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:23.085582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dbbdc-cba4-4175-824a-7fb84eeeb3ef None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:23.085978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dbbdc-cba4-4175-824a-7fb84eeeb3ef None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:23.086256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dbbdc-cba4-4175-824a-7fb84eeeb3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:23.106767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4e2dbbdc-cba4-4175-824a-7fb84eeeb3ef None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:23.107993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4e2dbbdc-cba4-4175-824a-7fb84eeeb3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:23.112573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40f63b-58c3-4e8c-a589-0b90511a3e05 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:23.130736 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:23.142966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:23.178021 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1093/2191] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:23.185770 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff03f5-9ed3-4764-9e11-9ca67bfe85ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:23.186400 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff03f5-9ed3-4764-9e11-9ca67bfe85ff service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:23.186675 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff03f5-9ed3-4764-9e11-9ca67bfe85ff service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:23.186942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff03f5-9ed3-4764-9e11-9ca67bfe85ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:23.189217 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2dbbdc-cba4-4175-824a-7fb84eeeb3ef None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:23.190152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5eff03f5-9ed3-4764-9e11-9ca67bfe85ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:23.190563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4e2dbbdc-cba4-4175-824a-7fb84eeeb3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:23.275434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff03f5-9ed3-4764-9e11-9ca67bfe85ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:23.277364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eff03f5-9ed3-4764-9e11-9ca67bfe85ff service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:23.283823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2dbbdc-cba4-4175-824a-7fb84eeeb3ef None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:23.289788 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97db19c-c5fa-430b-8de4-83584600465d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:23.290425 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97db19c-c5fa-430b-8de4-83584600465d service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:23.290697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97db19c-c5fa-430b-8de4-83584600465d service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:23.290969 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97db19c-c5fa-430b-8de4-83584600465d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:23.292905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1099/2192] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:23.302961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97db19c-c5fa-430b-8de4-83584600465d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:23.304682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97db19c-c5fa-430b-8de4-83584600465d service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:23.326319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1094/2193] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:23.861311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff03f5-9ed3-4764-9e11-9ca67bfe85ff service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:23.862913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5eff03f5-9ed3-4764-9e11-9ca67bfe85ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:23.880503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eff03f5-9ed3-4764-9e11-9ca67bfe85ff service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:23.887325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb1449-3a1d-41c6-bc24-e8736416d890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:23.888950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb1449-3a1d-41c6-bc24-e8736416d890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:23.889331 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb1449-3a1d-41c6-bc24-e8736416d890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:23.889699 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb1449-3a1d-41c6-bc24-e8736416d890 admin admin] PATH_INFO: `/v3/projects/d488e48d8712453a89b4118b6825fcf2` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:23.892350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb1449-3a1d-41c6-bc24-e8736416d890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d488e48d8712453a89b4118b6825fcf2)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:23.894533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb1449-3a1d-41c6-bc24-e8736416d890 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:24.021122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5feb1449-3a1d-41c6-bc24-e8736416d890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd488e48d8712453a89b4118b6825fcf2', 'request_id': 'req-5feb1449-3a1d-41c6-bc24-e8736416d890'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:24.032047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5feb1449-3a1d-41c6-bc24-e8736416d890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd488e48d8712453a89b4118b6825fcf2', 'request_id': 'req-5feb1449-3a1d-41c6-bc24-e8736416d890'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:24.042014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5feb1449-3a1d-41c6-bc24-e8736416d890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd488e48d8712453a89b4118b6825fcf2', 'request_id': 'req-5feb1449-3a1d-41c6-bc24-e8736416d890'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:24.054998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5feb1449-3a1d-41c6-bc24-e8736416d890 admin admin] The token cache is being invalidate because project d488e48d8712453a89b4118b6825fcf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:24.055289 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5feb1449-3a1d-41c6-bc24-e8736416d890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:24.055988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5feb1449-3a1d-41c6-bc24-e8736416d890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:24.061622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1100/2194] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:23 2022] DELETE /identity/v3/projects/d488e48d8712453a89b4118b6825fcf2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:24.063550 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fc125-ad2a-452e-81e6-cbdd4428d76f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.064019 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fc125-ad2a-452e-81e6-cbdd4428d76f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.064112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fc125-ad2a-452e-81e6-cbdd4428d76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.096828 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c74fc125-ad2a-452e-81e6-cbdd4428d76f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:24.098408 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c74fc125-ad2a-452e-81e6-cbdd4428d76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:24.189685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1095/2195] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:24.198343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bcb90-7376-4376-8551-a875646a2aeb None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.198656 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bcb90-7376-4376-8551-a875646a2aeb None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.198978 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bcb90-7376-4376-8551-a875646a2aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.226073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-961bcb90-7376-4376-8551-a875646a2aeb None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:24.227361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-961bcb90-7376-4376-8551-a875646a2aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:24.243090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1101/2196] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:24.244130 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fc125-ad2a-452e-81e6-cbdd4428d76f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:24.257754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fc125-ad2a-452e-81e6-cbdd4428d76f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:24.262764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb3e8d-8153-404a-b1a2-26a4c72035cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:24.263572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb3e8d-8153-404a-b1a2-26a4c72035cf admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.264045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb3e8d-8153-404a-b1a2-26a4c72035cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.264387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb3e8d-8153-404a-b1a2-26a4c72035cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.265510 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb3e8d-8153-404a-b1a2-26a4c72035cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:24.265840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb3e8d-8153-404a-b1a2-26a4c72035cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:24.267971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb3e8d-8153-404a-b1a2-26a4c72035cf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:24.282667 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1096/2197] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:20:24.287560 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bcb90-7376-4376-8551-a875646a2aeb None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:24.302830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bcb90-7376-4376-8551-a875646a2aeb None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:24.310908 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e6e8c-fe87-468d-ad06-100e9450e244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:24.311537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e6e8c-fe87-468d-ad06-100e9450e244 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.311807 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e6e8c-fe87-468d-ad06-100e9450e244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.312073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e6e8c-fe87-468d-ad06-100e9450e244 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.313862 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e6e8c-fe87-468d-ad06-100e9450e244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:24.315625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e6e8c-fe87-468d-ad06-100e9450e244 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:24.365135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1102/2198] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:24 2022] POST /identity/v3/projects => generated 377 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:24.368108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb3e8d-8153-404a-b1a2-26a4c72035cf admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:24.389398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb3e8d-8153-404a-b1a2-26a4c72035cf admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:24.492327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd46d6-d733-4e70-b39f-9d94cd9275a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:24.493189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd46d6-d733-4e70-b39f-9d94cd9275a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.493471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd46d6-d733-4e70-b39f-9d94cd9275a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.493737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd46d6-d733-4e70-b39f-9d94cd9275a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.494523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd46d6-d733-4e70-b39f-9d94cd9275a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:24.498429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd46d6-d733-4e70-b39f-9d94cd9275a5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:24.522611 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:24.549574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1097/2199] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:24 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:24.558048 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e6e8c-fe87-468d-ad06-100e9450e244 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:24.580046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e6e8c-fe87-468d-ad06-100e9450e244 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:24.587947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488b105-44e9-42ec-bb2d-95afbdc4528e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:24.589923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488b105-44e9-42ec-bb2d-95afbdc4528e admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.591939 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488b105-44e9-42ec-bb2d-95afbdc4528e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.593901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488b105-44e9-42ec-bb2d-95afbdc4528e admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.595107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488b105-44e9-42ec-bb2d-95afbdc4528e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:24.598087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488b105-44e9-42ec-bb2d-95afbdc4528e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:24.611670 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:24.626443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd46d6-d733-4e70-b39f-9d94cd9275a5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:24.627109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1103/2200] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:24.639958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd46d6-d733-4e70-b39f-9d94cd9275a5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:24.645675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486830da-3786-47ed-887a-5905a81d6d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:24.646849 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486830da-3786-47ed-887a-5905a81d6d34 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.647329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486830da-3786-47ed-887a-5905a81d6d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.647764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486830da-3786-47ed-887a-5905a81d6d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.648778 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486830da-3786-47ed-887a-5905a81d6d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:24.651125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486830da-3786-47ed-887a-5905a81d6d34 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:24.664400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1098/2201] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:24.669343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488b105-44e9-42ec-bb2d-95afbdc4528e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:24.687574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488b105-44e9-42ec-bb2d-95afbdc4528e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:24.697058 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7be05f-c3d9-4fe0-ba3d-5c8f27ed077d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:24.697860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7be05f-c3d9-4fe0-ba3d-5c8f27ed077d admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.698212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7be05f-c3d9-4fe0-ba3d-5c8f27ed077d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.698721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7be05f-c3d9-4fe0-ba3d-5c8f27ed077d admin admin] PATH_INFO: `/v3/projects/1d52492061fd4f37a6a3b538f720eeb7/users/8733b36fe7e34ec2b88031ac8752ca8a/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.741509 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7be05f-c3d9-4fe0-ba3d-5c8f27ed077d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d52492061fd4f37a6a3b538f720eeb7, user_id=8733b36fe7e34ec2b88031ac8752ca8a, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:24.744254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7be05f-c3d9-4fe0-ba3d-5c8f27ed077d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:24.776067 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d026d-9022-4aab-8e79-adf8150bf2b9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:24.776343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d026d-9022-4aab-8e79-adf8150bf2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:24.776673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d026d-9022-4aab-8e79-adf8150bf2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:24.780216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1104/2202] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:24 2022] PUT /identity/v3/projects/1d52492061fd4f37a6a3b538f720eeb7/users/8733b36fe7e34ec2b88031ac8752ca8a/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:24.815673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-998d026d-9022-4aab-8e79-adf8150bf2b9 None None] MFA Rules not processed for user `8733b36fe7e34ec2b88031ac8752ca8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:24.817532 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-998d026d-9022-4aab-8e79-adf8150bf2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:24.944187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1099/2203] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:24.957730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7be05f-c3d9-4fe0-ba3d-5c8f27ed077d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:24.960376 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5b7be05f-c3d9-4fe0-ba3d-5c8f27ed077d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:25.081764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7be05f-c3d9-4fe0-ba3d-5c8f27ed077d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:25.082454 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db287624-2503-4d55-9ef0-d006a895aff2 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:25.083285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db287624-2503-4d55-9ef0-d006a895aff2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:25.083727 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db287624-2503-4d55-9ef0-d006a895aff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:25.086704 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824e8ef-d02b-461e-813b-1758dffc89c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:25.087589 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e8ef-d02b-461e-813b-1758dffc89c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:25.088053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e8ef-d02b-461e-813b-1758dffc89c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:25.088384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824e8ef-d02b-461e-813b-1758dffc89c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:25.100631 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824e8ef-d02b-461e-813b-1758dffc89c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:25.102378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824e8ef-d02b-461e-813b-1758dffc89c8 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:25.105766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-db287624-2503-4d55-9ef0-d006a895aff2 None None] MFA Rules not processed for user `3077d51be16e40d9bb70b984583c0f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:25.107181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-db287624-2503-4d55-9ef0-d006a895aff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:25.118809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1105/2204] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:20:25.218857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1100/2205] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:20:29.374821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699c8e1-5f58-487d-8ef6-8758c9a87154 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:29.375678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699c8e1-5f58-487d-8ef6-8758c9a87154 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:29.375954 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699c8e1-5f58-487d-8ef6-8758c9a87154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:29.398154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d699c8e1-5f58-487d-8ef6-8758c9a87154 None None] MFA Rules not processed for user `159528b6ad49481f86bfbbac6187d584`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:29.399420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d699c8e1-5f58-487d-8ef6-8758c9a87154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:29.464931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1106/2206] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:35.199241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=64 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:39.456412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:50.557249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db287624-2503-4d55-9ef0-d006a895aff2 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:50.558801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-db287624-2503-4d55-9ef0-d006a895aff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:50.627022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db287624-2503-4d55-9ef0-d006a895aff2 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:50.631783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:50.632474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:50.632858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:50.633134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] PATH_INFO: `/v3/users/2145b870f54f4c71bd02c0153cb9bab6` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:50.634261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2145b870f54f4c71bd02c0153cb9bab6)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:50.636166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:50.689638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2145b870f54f4c71bd02c0153cb9bab6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:50.696404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2145b870f54f4c71bd02c0153cb9bab6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:50.709335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2145b870f54f4c71bd02c0153cb9bab6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:50.721198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2145b870f54f4c71bd02c0153cb9bab6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:50.721952 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2145b870f54f4c71bd02c0153cb9bab6'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:50.728321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1101/2207] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:50 2022] DELETE /identity/v3/users/2145b870f54f4c71bd02c0153cb9bab6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:50.732930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823000b-f51c-4795-be36-b531c393d278 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:50.733444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823000b-f51c-4795-be36-b531c393d278 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:50.733444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823000b-f51c-4795-be36-b531c393d278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:50.756977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a823000b-f51c-4795-be36-b531c393d278 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:50.758412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a823000b-f51c-4795-be36-b531c393d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:50.819951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1107/2208] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:50.830045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:50.831687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:50.893478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1068d-6768-49da-ba61-88abba67406d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:50.897632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ec837-7adf-41ae-9040-ddc8de4b25f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:50.898482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ec837-7adf-41ae-9040-ddc8de4b25f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:50.898640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ec837-7adf-41ae-9040-ddc8de4b25f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:50.898827 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ec837-7adf-41ae-9040-ddc8de4b25f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:50.908026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ec837-7adf-41ae-9040-ddc8de4b25f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:50.909473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ec837-7adf-41ae-9040-ddc8de4b25f9 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:50.921579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1102/2209] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:51.203020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823000b-f51c-4795-be36-b531c393d278 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:51.204469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a823000b-f51c-4795-be36-b531c393d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:51.216506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823000b-f51c-4795-be36-b531c393d278 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:51.221478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:51.222142 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:51.222393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:51.222649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] PATH_INFO: `/v3/projects/404c226fd2d9486b804f6b2670e4a824` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:51.223786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] RBAC: Authorizing `identity:delete_project(project_id=404c226fd2d9486b804f6b2670e4a824)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:51.225754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:51.274543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '404c226fd2d9486b804f6b2670e4a824', 'request_id': 'req-f792db76-8938-4f44-a6aa-89a139dab12d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:51.283625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '404c226fd2d9486b804f6b2670e4a824', 'request_id': 'req-f792db76-8938-4f44-a6aa-89a139dab12d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:51.290809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '404c226fd2d9486b804f6b2670e4a824', 'request_id': 'req-f792db76-8938-4f44-a6aa-89a139dab12d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:51.302523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] The token cache is being invalidate because project 404c226fd2d9486b804f6b2670e4a824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:51.302811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:51.303549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:51.305307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1108/2210] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:51 2022] DELETE /identity/v3/projects/404c226fd2d9486b804f6b2670e4a824 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:51.309496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ec837-7adf-41ae-9040-ddc8de4b25f9 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:51.319353 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ec837-7adf-41ae-9040-ddc8de4b25f9 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:51.339573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:51.383761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a39d0-df9e-4fd6-94c7-f6421bc9b661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:51.383761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a39d0-df9e-4fd6-94c7-f6421bc9b661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:51.383761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a39d0-df9e-4fd6-94c7-f6421bc9b661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:51.383761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a39d0-df9e-4fd6-94c7-f6421bc9b661 admin admin] PATH_INFO: `/v3/users/a05b778a57e54aeb840c33f623c35f95` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:51.385026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a39d0-df9e-4fd6-94c7-f6421bc9b661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a05b778a57e54aeb840c33f623c35f95)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:51.386934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a39d0-df9e-4fd6-94c7-f6421bc9b661 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:51.443683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3a6a39d0-df9e-4fd6-94c7-f6421bc9b661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a05b778a57e54aeb840c33f623c35f95'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:51.451028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3a6a39d0-df9e-4fd6-94c7-f6421bc9b661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a05b778a57e54aeb840c33f623c35f95'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:51.466851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3a6a39d0-df9e-4fd6-94c7-f6421bc9b661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a05b778a57e54aeb840c33f623c35f95'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:51.477068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3a6a39d0-df9e-4fd6-94c7-f6421bc9b661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a05b778a57e54aeb840c33f623c35f95'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:51.478027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3a6a39d0-df9e-4fd6-94c7-f6421bc9b661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a05b778a57e54aeb840c33f623c35f95'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:51.484982 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1103/2211] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:20:51 2022] DELETE /identity/v3/users/a05b778a57e54aeb840c33f623c35f95 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:51.846653 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:51.848099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:51.862033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792db76-8938-4f44-a6aa-89a139dab12d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:51.921802 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf1a21-f981-44db-969d-583df290e5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:51.922436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf1a21-f981-44db-969d-583df290e5ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:51.922711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf1a21-f981-44db-969d-583df290e5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:51.922982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf1a21-f981-44db-969d-583df290e5ff admin admin] PATH_INFO: `/v3/projects/a85a3fc1f44b41cb8ded83154b3c12f5` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:51.924100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf1a21-f981-44db-969d-583df290e5ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=a85a3fc1f44b41cb8ded83154b3c12f5)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:51.926043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf1a21-f981-44db-969d-583df290e5ff admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:51.959463 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:51.983542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8abf1a21-f981-44db-969d-583df290e5ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a85a3fc1f44b41cb8ded83154b3c12f5', 'request_id': 'req-8abf1a21-f981-44db-969d-583df290e5ff'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:51.993306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8abf1a21-f981-44db-969d-583df290e5ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a85a3fc1f44b41cb8ded83154b3c12f5', 'request_id': 'req-8abf1a21-f981-44db-969d-583df290e5ff'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:51.999685 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8abf1a21-f981-44db-969d-583df290e5ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a85a3fc1f44b41cb8ded83154b3c12f5', 'request_id': 'req-8abf1a21-f981-44db-969d-583df290e5ff'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:52.010131 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8abf1a21-f981-44db-969d-583df290e5ff admin admin] The token cache is being invalidate because project a85a3fc1f44b41cb8ded83154b3c12f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:20:52.010399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8abf1a21-f981-44db-969d-583df290e5ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:52.011019 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-8abf1a21-f981-44db-969d-583df290e5ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:20:52.012499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1109/2212] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:20:51 2022] DELETE /identity/v3/projects/a85a3fc1f44b41cb8ded83154b3c12f5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:52.017550 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26388c-45f8-4067-86a9-9751926ad828 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.018008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26388c-45f8-4067-86a9-9751926ad828 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.018008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26388c-45f8-4067-86a9-9751926ad828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.037695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7f26388c-45f8-4067-86a9-9751926ad828 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:52.038979 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7f26388c-45f8-4067-86a9-9751926ad828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:52.114229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1104/2213] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:52.123836 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17901be-af6c-4ea3-9276-b9e3476dfd96 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.124118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17901be-af6c-4ea3-9276-b9e3476dfd96 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.124383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17901be-af6c-4ea3-9276-b9e3476dfd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.144889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b17901be-af6c-4ea3-9276-b9e3476dfd96 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:52.146059 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b17901be-af6c-4ea3-9276-b9e3476dfd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:52.155430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1110/2214] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:52.159832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26388c-45f8-4067-86a9-9751926ad828 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:52.170101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26388c-45f8-4067-86a9-9751926ad828 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:52.175395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123abb4-929c-490d-b715-0220160b4180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:52.175989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123abb4-929c-490d-b715-0220160b4180 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.176258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123abb4-929c-490d-b715-0220160b4180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.176523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123abb4-929c-490d-b715-0220160b4180 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.177210 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123abb4-929c-490d-b715-0220160b4180 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:20:52.177501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123abb4-929c-490d-b715-0220160b4180 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:52.179092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123abb4-929c-490d-b715-0220160b4180 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:52.192557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1105/2215] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:20:52.197776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17901be-af6c-4ea3-9276-b9e3476dfd96 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:52.210162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17901be-af6c-4ea3-9276-b9e3476dfd96 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:52.216293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2586cc-7a2a-4b41-8944-140fd2c9dc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:52.216942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2586cc-7a2a-4b41-8944-140fd2c9dc12 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.217220 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2586cc-7a2a-4b41-8944-140fd2c9dc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.217486 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2586cc-7a2a-4b41-8944-140fd2c9dc12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.218125 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2586cc-7a2a-4b41-8944-140fd2c9dc12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:52.220105 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2586cc-7a2a-4b41-8944-140fd2c9dc12 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:52.252254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1111/2216] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:52 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:52.256580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123abb4-929c-490d-b715-0220160b4180 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:52.267385 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123abb4-929c-490d-b715-0220160b4180 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:52.293078 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:52.327253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98392d-02c1-4d01-9cf4-b7b2ea77d538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:52.327864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98392d-02c1-4d01-9cf4-b7b2ea77d538 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.328166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98392d-02c1-4d01-9cf4-b7b2ea77d538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.328449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98392d-02c1-4d01-9cf4-b7b2ea77d538 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.329105 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98392d-02c1-4d01-9cf4-b7b2ea77d538 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:52.330995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98392d-02c1-4d01-9cf4-b7b2ea77d538 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:52.358448 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1106/2217] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:52 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:52.363512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2586cc-7a2a-4b41-8944-140fd2c9dc12 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:52.374616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2586cc-7a2a-4b41-8944-140fd2c9dc12 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:52.379811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c1e5d-69cc-4fe7-8e7c-a2aea8e585d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:52.380406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c1e5d-69cc-4fe7-8e7c-a2aea8e585d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.380711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c1e5d-69cc-4fe7-8e7c-a2aea8e585d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.380978 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c1e5d-69cc-4fe7-8e7c-a2aea8e585d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.381627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c1e5d-69cc-4fe7-8e7c-a2aea8e585d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:52.383362 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c1e5d-69cc-4fe7-8e7c-a2aea8e585d9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:52.393523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1112/2218] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:52.397653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98392d-02c1-4d01-9cf4-b7b2ea77d538 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:52.408003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98392d-02c1-4d01-9cf4-b7b2ea77d538 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:52.413000 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cdd74-713e-4751-8aa5-a5cad314c75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:52.413619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cdd74-713e-4751-8aa5-a5cad314c75e admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.414038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cdd74-713e-4751-8aa5-a5cad314c75e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.414311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cdd74-713e-4751-8aa5-a5cad314c75e admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.414970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cdd74-713e-4751-8aa5-a5cad314c75e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:52.416623 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cdd74-713e-4751-8aa5-a5cad314c75e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:52.427541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1107/2219] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:52.432231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c1e5d-69cc-4fe7-8e7c-a2aea8e585d9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:52.442475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c1e5d-69cc-4fe7-8e7c-a2aea8e585d9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:52.447482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d25a57-ee91-42a6-a9d6-60481b22b54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:52.448134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d25a57-ee91-42a6-a9d6-60481b22b54b admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.448407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d25a57-ee91-42a6-a9d6-60481b22b54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.448718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d25a57-ee91-42a6-a9d6-60481b22b54b admin admin] PATH_INFO: `/v3/projects/3c7a5027ca3847cebc847a75f0729234/users/9fd92811f43a4fbcb1345b41980f41b4/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.463595 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d25a57-ee91-42a6-a9d6-60481b22b54b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7a5027ca3847cebc847a75f0729234, user_id=9fd92811f43a4fbcb1345b41980f41b4, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:52.466294 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d25a57-ee91-42a6-a9d6-60481b22b54b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:52.481373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1113/2220] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:52 2022] PUT /identity/v3/projects/3c7a5027ca3847cebc847a75f0729234/users/9fd92811f43a4fbcb1345b41980f41b4/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:20:52.492841 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06686227-b019-4bab-870b-9df486712a62 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.493120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06686227-b019-4bab-870b-9df486712a62 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.493386 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06686227-b019-4bab-870b-9df486712a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.521725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-06686227-b019-4bab-870b-9df486712a62 None None] MFA Rules not processed for user `9fd92811f43a4fbcb1345b41980f41b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:52.522855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-06686227-b019-4bab-870b-9df486712a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:52.594549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1108/2221] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:52 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:52.601046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d25a57-ee91-42a6-a9d6-60481b22b54b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:52.666708 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d25a57-ee91-42a6-a9d6-60481b22b54b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:52.671678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802ec49-bb21-44a2-95bd-2f5a87371abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:52.672291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ec49-bb21-44a2-95bd-2f5a87371abe admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.672747 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ec49-bb21-44a2-95bd-2f5a87371abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.672845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9802ec49-bb21-44a2-95bd-2f5a87371abe admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.673471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802ec49-bb21-44a2-95bd-2f5a87371abe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:52.675374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9802ec49-bb21-44a2-95bd-2f5a87371abe admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:52.706499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1114/2222] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:20:52 2022] POST /identity/v3/projects => generated 399 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:52.710843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06686227-b019-4bab-870b-9df486712a62 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:52.722391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06686227-b019-4bab-870b-9df486712a62 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:52.785794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17e914-cae5-4b1c-8c7f-9e761c2b0a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:52.786312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17e914-cae5-4b1c-8c7f-9e761c2b0a9a admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.786612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17e914-cae5-4b1c-8c7f-9e761c2b0a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.786866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17e914-cae5-4b1c-8c7f-9e761c2b0a9a admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.787657 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17e914-cae5-4b1c-8c7f-9e761c2b0a9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:52.789742 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17e914-cae5-4b1c-8c7f-9e761c2b0a9a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:52.816624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1109/2223] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:20:52 2022] POST /identity/v3/users => generated 306 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:20:52.818861 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802ec49-bb21-44a2-95bd-2f5a87371abe admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:52.829602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9802ec49-bb21-44a2-95bd-2f5a87371abe admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:52.835541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3929d-0901-4151-9690-5291c7f7aa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:52.836809 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3929d-0901-4151-9690-5291c7f7aa80 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.837116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3929d-0901-4151-9690-5291c7f7aa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.837507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3929d-0901-4151-9690-5291c7f7aa80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.838149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3929d-0901-4151-9690-5291c7f7aa80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:52.839752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3929d-0901-4151-9690-5291c7f7aa80 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:52.850964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1115/2224] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:52.855761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17e914-cae5-4b1c-8c7f-9e761c2b0a9a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:52.872039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17e914-cae5-4b1c-8c7f-9e761c2b0a9a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:52.877608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a216c-c569-4fbe-a68c-ff4e5ef16296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:52.878313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a216c-c569-4fbe-a68c-ff4e5ef16296 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.878587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a216c-c569-4fbe-a68c-ff4e5ef16296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.878856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a216c-c569-4fbe-a68c-ff4e5ef16296 admin admin] PATH_INFO: `/v3/projects/97a579740dab46699e50f9f5ec8e044c/users/c3df277a8a144cde8c823f231526815c/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.894540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a216c-c569-4fbe-a68c-ff4e5ef16296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a579740dab46699e50f9f5ec8e044c, user_id=c3df277a8a144cde8c823f231526815c, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:52.897228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a216c-c569-4fbe-a68c-ff4e5ef16296 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:52.912934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1110/2225] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:52 2022] PUT /identity/v3/projects/97a579740dab46699e50f9f5ec8e044c/users/c3df277a8a144cde8c823f231526815c/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:52.917691 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3929d-0901-4151-9690-5291c7f7aa80 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:52.927944 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3929d-0901-4151-9690-5291c7f7aa80 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:52.950657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:52.991488 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f0b82-21f0-4c3c-aa7b-10d91bc47ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:52.992341 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f0b82-21f0-4c3c-aa7b-10d91bc47ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:52.992730 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f0b82-21f0-4c3c-aa7b-10d91bc47ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:52.993004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f0b82-21f0-4c3c-aa7b-10d91bc47ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:52.993637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f0b82-21f0-4c3c-aa7b-10d91bc47ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:52.995232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f0b82-21f0-4c3c-aa7b-10d91bc47ef6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:53.004836 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1116/2226] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:20:53.010522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a216c-c569-4fbe-a68c-ff4e5ef16296 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:53.019849 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:20:53.028693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a216c-c569-4fbe-a68c-ff4e5ef16296 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:53.038277 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0855b8-d0a4-4f3f-a11a-194a84cb399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:53.039068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0855b8-d0a4-4f3f-a11a-194a84cb399e admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:53.039338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0855b8-d0a4-4f3f-a11a-194a84cb399e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:53.039602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0855b8-d0a4-4f3f-a11a-194a84cb399e admin admin] PATH_INFO: `/v3/projects/97a579740dab46699e50f9f5ec8e044c/users/c3df277a8a144cde8c823f231526815c/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:53.041610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0855b8-d0a4-4f3f-a11a-194a84cb399e admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a579740dab46699e50f9f5ec8e044c, user_id=c3df277a8a144cde8c823f231526815c, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:53.044248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0855b8-d0a4-4f3f-a11a-194a84cb399e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:53.055380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1111/2227] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:20:53 2022] PUT /identity/v3/projects/97a579740dab46699e50f9f5ec8e044c/users/c3df277a8a144cde8c823f231526815c/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:20:53.065744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821851ce-e129-4a5d-be53-c54643d50ac2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:53.066026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821851ce-e129-4a5d-be53-c54643d50ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:53.066291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821851ce-e129-4a5d-be53-c54643d50ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:53.097420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-821851ce-e129-4a5d-be53-c54643d50ac2 None None] MFA Rules not processed for user `c3df277a8a144cde8c823f231526815c`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:20:53.098627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-821851ce-e129-4a5d-be53-c54643d50ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:53.177051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1117/2228] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:20:53 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:20:53.188323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0855b8-d0a4-4f3f-a11a-194a84cb399e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:53.190204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-af0855b8-d0a4-4f3f-a11a-194a84cb399e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:20:53.258212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0855b8-d0a4-4f3f-a11a-194a84cb399e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:53.261957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90eef-6c50-455f-bf7c-cd8ebcccdfb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:53.262560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90eef-6c50-455f-bf7c-cd8ebcccdfb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:53.262847 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90eef-6c50-455f-bf7c-cd8ebcccdfb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:53.263115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e90eef-6c50-455f-bf7c-cd8ebcccdfb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:53.272311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e90eef-6c50-455f-bf7c-cd8ebcccdfb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:53.274177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e90eef-6c50-455f-bf7c-cd8ebcccdfb9 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:53.283939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1112/2229] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:53 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:20:53.371094 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821851ce-e129-4a5d-be53-c54643d50ac2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:20:53.380926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821851ce-e129-4a5d-be53-c54643d50ac2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:20:53.385373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1c54f-d4c5-4466-9bac-ce473045d6c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:20:53.389614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1c54f-d4c5-4466-9bac-ce473045d6c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:20:53.389614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1c54f-d4c5-4466-9bac-ce473045d6c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:20:53.389614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1c54f-d4c5-4466-9bac-ce473045d6c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:20:53.448085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1c54f-d4c5-4466-9bac-ce473045d6c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:20:53.449557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf1c54f-d4c5-4466-9bac-ce473045d6c2 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:20:53.458403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1118/2230] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:20:53 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:03.279615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:03.454379 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=80 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:08.402007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90eef-6c50-455f-bf7c-cd8ebcccdfb9 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:08.466431 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e90eef-6c50-455f-bf7c-cd8ebcccdfb9 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:08.470981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:08.471610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:08.471884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:08.472152 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] PATH_INFO: `/v3/users/9fd92811f43a4fbcb1345b41980f41b4` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:08.473301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fd92811f43a4fbcb1345b41980f41b4)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:08.475243 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:08.526900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fd92811f43a4fbcb1345b41980f41b4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:08.533805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fd92811f43a4fbcb1345b41980f41b4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:08.547687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fd92811f43a4fbcb1345b41980f41b4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:08.559112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fd92811f43a4fbcb1345b41980f41b4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:08.559813 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fd92811f43a4fbcb1345b41980f41b4'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:08.566204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1113/2231] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:08 2022] DELETE /identity/v3/users/9fd92811f43a4fbcb1345b41980f41b4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:08.571246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8162555-40b0-4778-96f2-c8b22edb9ba4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:08.571759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8162555-40b0-4778-96f2-c8b22edb9ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:08.571759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8162555-40b0-4778-96f2-c8b22edb9ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:08.591473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f8162555-40b0-4778-96f2-c8b22edb9ba4 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:08.592778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f8162555-40b0-4778-96f2-c8b22edb9ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:08.665118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1119/2232] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:08.672688 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:08.674787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:08.736376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfef57-afa2-430c-88d1-6afc2be3ce94 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:08.740834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a5649-4a70-46b3-abac-6f669eff6e06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:08.741442 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a5649-4a70-46b3-abac-6f669eff6e06 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:08.741713 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a5649-4a70-46b3-abac-6f669eff6e06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:08.741977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a5649-4a70-46b3-abac-6f669eff6e06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:08.751683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a5649-4a70-46b3-abac-6f669eff6e06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:08.753115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a5649-4a70-46b3-abac-6f669eff6e06 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:08.763433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1114/2233] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:09.052565 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8162555-40b0-4778-96f2-c8b22edb9ba4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:09.054099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f8162555-40b0-4778-96f2-c8b22edb9ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:09.066819 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8162555-40b0-4778-96f2-c8b22edb9ba4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:09.072091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42259a-c2b0-48d1-a856-56e197784e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:09.072758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42259a-c2b0-48d1-a856-56e197784e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:09.073066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42259a-c2b0-48d1-a856-56e197784e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:09.073315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42259a-c2b0-48d1-a856-56e197784e87 admin admin] PATH_INFO: `/v3/projects/3c7a5027ca3847cebc847a75f0729234` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:09.075497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42259a-c2b0-48d1-a856-56e197784e87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c7a5027ca3847cebc847a75f0729234)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:09.081771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42259a-c2b0-48d1-a856-56e197784e87 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:09.136195 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1f42259a-c2b0-48d1-a856-56e197784e87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c7a5027ca3847cebc847a75f0729234', 'request_id': 'req-1f42259a-c2b0-48d1-a856-56e197784e87'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.147899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1f42259a-c2b0-48d1-a856-56e197784e87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c7a5027ca3847cebc847a75f0729234', 'request_id': 'req-1f42259a-c2b0-48d1-a856-56e197784e87'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.157734 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1f42259a-c2b0-48d1-a856-56e197784e87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c7a5027ca3847cebc847a75f0729234', 'request_id': 'req-1f42259a-c2b0-48d1-a856-56e197784e87'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.171227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1f42259a-c2b0-48d1-a856-56e197784e87 admin admin] The token cache is being invalidate because project 3c7a5027ca3847cebc847a75f0729234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:09.171647 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1f42259a-c2b0-48d1-a856-56e197784e87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.172230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-1f42259a-c2b0-48d1-a856-56e197784e87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.174088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1120/2234] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:09 2022] DELETE /identity/v3/projects/3c7a5027ca3847cebc847a75f0729234 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:09.177920 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a5649-4a70-46b3-abac-6f669eff6e06 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:09.188786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a5649-4a70-46b3-abac-6f669eff6e06 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:09.213459 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:09.259384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6003b-782a-4722-8400-a70239872082 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:09.261042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6003b-782a-4722-8400-a70239872082 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:09.262101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6003b-782a-4722-8400-a70239872082 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:09.263903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1121/2235] 10.222.0.75 () {54 vars in 961 bytes} [Fri Sep 23 15:21:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:21:09.269053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:09.269707 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:09.269985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:09.270260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] PATH_INFO: `/v3/users/c3df277a8a144cde8c823f231526815c` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:09.271467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3df277a8a144cde8c823f231526815c)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:09.271800 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12996a1c-2c66-4307-b9e1-ad16c1d82657 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:09.272181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12996a1c-2c66-4307-b9e1-ad16c1d82657 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:09.272607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12996a1c-2c66-4307-b9e1-ad16c1d82657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:09.273608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:09.324614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-12996a1c-2c66-4307-b9e1-ad16c1d82657 None None] MFA Rules not processed for user `2b8be1ec4a7a45b88d84bd9c48fab407`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:09.324614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-12996a1c-2c66-4307-b9e1-ad16c1d82657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:09.345805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3df277a8a144cde8c823f231526815c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.356147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3df277a8a144cde8c823f231526815c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.397142 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3df277a8a144cde8c823f231526815c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.413405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3df277a8a144cde8c823f231526815c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.415298 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3df277a8a144cde8c823f231526815c'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.421370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1122/2236] 10.222.0.75 () {58 vars in 1058 bytes} [Fri Sep 23 15:21:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:09.424742 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1115/2237] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:09 2022] DELETE /identity/v3/users/c3df277a8a144cde8c823f231526815c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:09.444635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12996a1c-2c66-4307-b9e1-ad16c1d82657 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:09.457698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12996a1c-2c66-4307-b9e1-ad16c1d82657 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:09.479250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:09.537977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17afe689-00a6-4d9f-ba54-2be07cda4b07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:09.541984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afe689-00a6-4d9f-ba54-2be07cda4b07 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:09.542297 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afe689-00a6-4d9f-ba54-2be07cda4b07 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:09.542569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17afe689-00a6-4d9f-ba54-2be07cda4b07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:09.544769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-17afe689-00a6-4d9f-ba54-2be07cda4b07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:09.563707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17afe689-00a6-4d9f-ba54-2be07cda4b07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:09.565698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17afe689-00a6-4d9f-ba54-2be07cda4b07 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:09.582078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1123/2238] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:09.840001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:09.842873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:09.919119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed4532-0139-492d-87c6-547345581a1f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:09.923542 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d614b-78d8-4a0d-a417-e0e9396065d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:09.924169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d614b-78d8-4a0d-a417-e0e9396065d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:09.924436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d614b-78d8-4a0d-a417-e0e9396065d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:09.924738 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d614b-78d8-4a0d-a417-e0e9396065d7 admin admin] PATH_INFO: `/v3/projects/97a579740dab46699e50f9f5ec8e044c` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:09.925831 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d614b-78d8-4a0d-a417-e0e9396065d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97a579740dab46699e50f9f5ec8e044c)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:09.927696 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d614b-78d8-4a0d-a417-e0e9396065d7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:09.976894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:09.981153 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-176d614b-78d8-4a0d-a417-e0e9396065d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97a579740dab46699e50f9f5ec8e044c', 'request_id': 'req-176d614b-78d8-4a0d-a417-e0e9396065d7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.988740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-176d614b-78d8-4a0d-a417-e0e9396065d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97a579740dab46699e50f9f5ec8e044c', 'request_id': 'req-176d614b-78d8-4a0d-a417-e0e9396065d7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:09.994834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-176d614b-78d8-4a0d-a417-e0e9396065d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97a579740dab46699e50f9f5ec8e044c', 'request_id': 'req-176d614b-78d8-4a0d-a417-e0e9396065d7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:10.009082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-176d614b-78d8-4a0d-a417-e0e9396065d7 admin admin] The token cache is being invalidate because project 97a579740dab46699e50f9f5ec8e044c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:10.009368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-176d614b-78d8-4a0d-a417-e0e9396065d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:10.010024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-176d614b-78d8-4a0d-a417-e0e9396065d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:10.012203 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1116/2239] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:09 2022] DELETE /identity/v3/projects/97a579740dab46699e50f9f5ec8e044c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:10.017272 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227b77e-3ec5-4397-b797-2ebffbc1dde3 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.017553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227b77e-3ec5-4397-b797-2ebffbc1dde3 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.017818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227b77e-3ec5-4397-b797-2ebffbc1dde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.037780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d227b77e-3ec5-4397-b797-2ebffbc1dde3 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:10.038900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d227b77e-3ec5-4397-b797-2ebffbc1dde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:10.099312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1124/2240] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:10.107297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0a15a-1b91-4fb1-a214-bfc3166e9c0d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.107578 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0a15a-1b91-4fb1-a214-bfc3166e9c0d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.107848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0a15a-1b91-4fb1-a214-bfc3166e9c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.129347 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-eac0a15a-1b91-4fb1-a214-bfc3166e9c0d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:10.131175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-eac0a15a-1b91-4fb1-a214-bfc3166e9c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:10.139808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1117/2241] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:10.144376 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227b77e-3ec5-4397-b797-2ebffbc1dde3 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:10.157224 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227b77e-3ec5-4397-b797-2ebffbc1dde3 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:10.163453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2a2f9-34ce-45ad-b3d4-47f311cb93d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:10.164049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2a2f9-34ce-45ad-b3d4-47f311cb93d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.164313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2a2f9-34ce-45ad-b3d4-47f311cb93d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.164573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2a2f9-34ce-45ad-b3d4-47f311cb93d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.165545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2a2f9-34ce-45ad-b3d4-47f311cb93d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:10.165839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2a2f9-34ce-45ad-b3d4-47f311cb93d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:10.168182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2a2f9-34ce-45ad-b3d4-47f311cb93d3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:10.180012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1125/2242] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:10.184621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0a15a-1b91-4fb1-a214-bfc3166e9c0d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:10.193747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0a15a-1b91-4fb1-a214-bfc3166e9c0d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:10.198931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cbf7d-f01a-4553-8243-16ccc2a5dc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:10.199525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cbf7d-f01a-4553-8243-16ccc2a5dc6d admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.199788 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cbf7d-f01a-4553-8243-16ccc2a5dc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.200051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cbf7d-f01a-4553-8243-16ccc2a5dc6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.200722 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9cbf7d-f01a-4553-8243-16ccc2a5dc6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:10.202597 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9cbf7d-f01a-4553-8243-16ccc2a5dc6d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:10.235120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1118/2243] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:10 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:10.240240 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2a2f9-34ce-45ad-b3d4-47f311cb93d3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:10.252819 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2a2f9-34ce-45ad-b3d4-47f311cb93d3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:10.317896 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303fbff-43df-4115-90f7-74cb62de312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:10.318534 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303fbff-43df-4115-90f7-74cb62de312c admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.318786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303fbff-43df-4115-90f7-74cb62de312c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.319057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303fbff-43df-4115-90f7-74cb62de312c admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.319696 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303fbff-43df-4115-90f7-74cb62de312c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:10.321679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303fbff-43df-4115-90f7-74cb62de312c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:10.346167 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1126/2244] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:10 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:10.351302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cbf7d-f01a-4553-8243-16ccc2a5dc6d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:10.365659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cbf7d-f01a-4553-8243-16ccc2a5dc6d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:10.373039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbbc89-1afd-4c6e-8e9f-6be92f6bc9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:10.374057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbbc89-1afd-4c6e-8e9f-6be92f6bc9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.374548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbbc89-1afd-4c6e-8e9f-6be92f6bc9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.374844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbbc89-1afd-4c6e-8e9f-6be92f6bc9e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.375494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbbc89-1afd-4c6e-8e9f-6be92f6bc9e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:10.377810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbbc89-1afd-4c6e-8e9f-6be92f6bc9e2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:10.393533 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1119/2245] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:10.399923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303fbff-43df-4115-90f7-74cb62de312c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:10.413955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303fbff-43df-4115-90f7-74cb62de312c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:10.420345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36175f8f-6081-4cb9-9fee-14d27519e9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:10.420994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36175f8f-6081-4cb9-9fee-14d27519e9ff admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.421256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36175f8f-6081-4cb9-9fee-14d27519e9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.421492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36175f8f-6081-4cb9-9fee-14d27519e9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.422217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36175f8f-6081-4cb9-9fee-14d27519e9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:10.423865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36175f8f-6081-4cb9-9fee-14d27519e9ff admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:10.441635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1127/2246] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:10.447254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbbc89-1afd-4c6e-8e9f-6be92f6bc9e2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:10.459796 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbbc89-1afd-4c6e-8e9f-6be92f6bc9e2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:10.467647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9688b43-f597-4b2b-85ee-727da5995cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:10.469370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9688b43-f597-4b2b-85ee-727da5995cf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.469370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9688b43-f597-4b2b-85ee-727da5995cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.469370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9688b43-f597-4b2b-85ee-727da5995cf2 admin admin] PATH_INFO: `/v3/projects/e549d0bfa74f4fbebc107476b3927ec8/users/d60b85461f954ed4b69c5e02d7d41538/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.490400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9688b43-f597-4b2b-85ee-727da5995cf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e549d0bfa74f4fbebc107476b3927ec8, user_id=d60b85461f954ed4b69c5e02d7d41538, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:10.493252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9688b43-f597-4b2b-85ee-727da5995cf2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:10.517922 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1120/2247] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:10 2022] PUT /identity/v3/projects/e549d0bfa74f4fbebc107476b3927ec8/users/d60b85461f954ed4b69c5e02d7d41538/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:10.531943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cda786-1b09-4232-beb6-a616da7fe4b7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.532221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cda786-1b09-4232-beb6-a616da7fe4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.532492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cda786-1b09-4232-beb6-a616da7fe4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.567322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-01cda786-1b09-4232-beb6-a616da7fe4b7 None None] MFA Rules not processed for user `d60b85461f954ed4b69c5e02d7d41538`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:10.568661 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-01cda786-1b09-4232-beb6-a616da7fe4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:10.576403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:10.657824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1128/2248] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:10.660945 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9688b43-f597-4b2b-85ee-727da5995cf2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:10.732523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9688b43-f597-4b2b-85ee-727da5995cf2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:10.737891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b7958-214f-4cff-9d03-8db723715462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:10.738514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b7958-214f-4cff-9d03-8db723715462 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.738789 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b7958-214f-4cff-9d03-8db723715462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.739056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b7958-214f-4cff-9d03-8db723715462 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.739696 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b7958-214f-4cff-9d03-8db723715462 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:10.742837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b7958-214f-4cff-9d03-8db723715462 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:10.777579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1121/2249] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:10 2022] POST /identity/v3/projects => generated 389 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:10.781949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cda786-1b09-4232-beb6-a616da7fe4b7 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:10.792943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cda786-1b09-4232-beb6-a616da7fe4b7 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:10.859291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f0aec-91c5-4105-8d32-2a87f0344b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:10.859895 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f0aec-91c5-4105-8d32-2a87f0344b3e admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.860180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f0aec-91c5-4105-8d32-2a87f0344b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.860445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f0aec-91c5-4105-8d32-2a87f0344b3e admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.861098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f0aec-91c5-4105-8d32-2a87f0344b3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:10.863084 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f0aec-91c5-4105-8d32-2a87f0344b3e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:10.889203 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1129/2250] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:10 2022] POST /identity/v3/users => generated 301 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:10.893086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b7958-214f-4cff-9d03-8db723715462 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:10.910658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b7958-214f-4cff-9d03-8db723715462 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:10.923578 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d721101-4ac9-4f5c-ba10-61afe2910492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:10.924629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d721101-4ac9-4f5c-ba10-61afe2910492 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.924629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d721101-4ac9-4f5c-ba10-61afe2910492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.924840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d721101-4ac9-4f5c-ba10-61afe2910492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.929193 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d721101-4ac9-4f5c-ba10-61afe2910492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:10.929193 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d721101-4ac9-4f5c-ba10-61afe2910492 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:10.940733 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1122/2251] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:10.945508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f0aec-91c5-4105-8d32-2a87f0344b3e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:10.966303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f0aec-91c5-4105-8d32-2a87f0344b3e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:10.973184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea12e5-f35b-41b0-ba8c-cfd19956f939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:10.973943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea12e5-f35b-41b0-ba8c-cfd19956f939 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:10.974225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea12e5-f35b-41b0-ba8c-cfd19956f939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:10.974490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea12e5-f35b-41b0-ba8c-cfd19956f939 admin admin] PATH_INFO: `/v3/projects/63cf9383f04d4e8994f1aa38f7bf2f1f/users/e748eebed0b54b2d8eaa4cb0bfd7f30a/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:10.996912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea12e5-f35b-41b0-ba8c-cfd19956f939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63cf9383f04d4e8994f1aa38f7bf2f1f, user_id=e748eebed0b54b2d8eaa4cb0bfd7f30a, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:10.999583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea12e5-f35b-41b0-ba8c-cfd19956f939 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.016766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1130/2252] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:10 2022] PUT /identity/v3/projects/63cf9383f04d4e8994f1aa38f7bf2f1f/users/e748eebed0b54b2d8eaa4cb0bfd7f30a/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:11.020928 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d721101-4ac9-4f5c-ba10-61afe2910492 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:11.031924 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d721101-4ac9-4f5c-ba10-61afe2910492 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:11.075962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:11.097963 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cc5d9-5ab9-473f-9e7f-08f30c18075f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:11.098492 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cc5d9-5ab9-473f-9e7f-08f30c18075f admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.098762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cc5d9-5ab9-473f-9e7f-08f30c18075f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.099030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cc5d9-5ab9-473f-9e7f-08f30c18075f admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.099699 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cc5d9-5ab9-473f-9e7f-08f30c18075f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:11.101371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cc5d9-5ab9-473f-9e7f-08f30c18075f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.112454 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1123/2253] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:11.119227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea12e5-f35b-41b0-ba8c-cfd19956f939 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:11.131620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea12e5-f35b-41b0-ba8c-cfd19956f939 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:11.141662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454e4f9-d4d7-4a24-8643-40c4cc5e4e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:11.141662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454e4f9-d4d7-4a24-8643-40c4cc5e4e56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.141662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454e4f9-d4d7-4a24-8643-40c4cc5e4e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.141662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454e4f9-d4d7-4a24-8643-40c4cc5e4e56 admin admin] PATH_INFO: `/v3/projects/63cf9383f04d4e8994f1aa38f7bf2f1f/users/e748eebed0b54b2d8eaa4cb0bfd7f30a/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.142138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f454e4f9-d4d7-4a24-8643-40c4cc5e4e56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63cf9383f04d4e8994f1aa38f7bf2f1f, user_id=e748eebed0b54b2d8eaa4cb0bfd7f30a, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:11.145637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f454e4f9-d4d7-4a24-8643-40c4cc5e4e56 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.159408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1131/2254] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:11 2022] PUT /identity/v3/projects/63cf9383f04d4e8994f1aa38f7bf2f1f/users/e748eebed0b54b2d8eaa4cb0bfd7f30a/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:11.172128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a15f84-3252-450e-b311-773aca79d9d6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.172408 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a15f84-3252-450e-b311-773aca79d9d6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.172707 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a15f84-3252-450e-b311-773aca79d9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.209223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-05a15f84-3252-450e-b311-773aca79d9d6 None None] MFA Rules not processed for user `e748eebed0b54b2d8eaa4cb0bfd7f30a`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:11.210627 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-05a15f84-3252-450e-b311-773aca79d9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:11.309888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1124/2255] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:11 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:11.321248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454e4f9-d4d7-4a24-8643-40c4cc5e4e56 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:11.323195 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f454e4f9-d4d7-4a24-8643-40c4cc5e4e56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:11.398610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454e4f9-d4d7-4a24-8643-40c4cc5e4e56 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:11.402857 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf133fd-623d-43f0-b7a6-9aed974d1de9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:11.403463 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf133fd-623d-43f0-b7a6-9aed974d1de9 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.403743 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf133fd-623d-43f0-b7a6-9aed974d1de9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.404018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf133fd-623d-43f0-b7a6-9aed974d1de9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.413572 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf133fd-623d-43f0-b7a6-9aed974d1de9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:11.415013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf133fd-623d-43f0-b7a6-9aed974d1de9 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.420458 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:11.425961 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1132/2256] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:11 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:11.579647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a15f84-3252-450e-b311-773aca79d9d6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:11.645575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a15f84-3252-450e-b311-773aca79d9d6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:11.652403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:11.653200 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.653496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.653787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] PATH_INFO: `/v3/users/d60b85461f954ed4b69c5e02d7d41538` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.655408 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d60b85461f954ed4b69c5e02d7d41538)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:11.657381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.710646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd60b85461f954ed4b69c5e02d7d41538'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:11.718031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd60b85461f954ed4b69c5e02d7d41538'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:11.726617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:11.733709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd60b85461f954ed4b69c5e02d7d41538'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:11.744805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd60b85461f954ed4b69c5e02d7d41538'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:11.746745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd60b85461f954ed4b69c5e02d7d41538'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:11.756283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1125/2257] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:11 2022] DELETE /identity/v3/users/d60b85461f954ed4b69c5e02d7d41538 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:11.760633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f97483-7fe6-423d-87bc-00ae55aa2939 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.761051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f97483-7fe6-423d-87bc-00ae55aa2939 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.762002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f97483-7fe6-423d-87bc-00ae55aa2939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.786369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-98f97483-7fe6-423d-87bc-00ae55aa2939 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:11.787617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-98f97483-7fe6-423d-87bc-00ae55aa2939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:11.859532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1133/2258] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:11.869852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:11.871904 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:11.932175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48ed59-f69a-4168-a1cc-79ec3e9d2392 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:11.936820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07876a52-af3f-4831-8de9-d8a5de8ea393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:11.937426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07876a52-af3f-4831-8de9-d8a5de8ea393 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:11.937697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07876a52-af3f-4831-8de9-d8a5de8ea393 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:11.937967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07876a52-af3f-4831-8de9-d8a5de8ea393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:11.948191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07876a52-af3f-4831-8de9-d8a5de8ea393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:11.950165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07876a52-af3f-4831-8de9-d8a5de8ea393 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:11.960236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1126/2259] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:12.251078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f97483-7fe6-423d-87bc-00ae55aa2939 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:12.252912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-98f97483-7fe6-423d-87bc-00ae55aa2939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:12.265667 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f97483-7fe6-423d-87bc-00ae55aa2939 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:12.270957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:12.271597 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:12.271869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:12.272143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] PATH_INFO: `/v3/projects/e549d0bfa74f4fbebc107476b3927ec8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:12.274393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e549d0bfa74f4fbebc107476b3927ec8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:12.276497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:12.326802 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e549d0bfa74f4fbebc107476b3927ec8', 'request_id': 'req-71577d9c-9ff3-4ea6-a3b6-18be9b824729'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.335317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e549d0bfa74f4fbebc107476b3927ec8', 'request_id': 'req-71577d9c-9ff3-4ea6-a3b6-18be9b824729'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.342173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e549d0bfa74f4fbebc107476b3927ec8', 'request_id': 'req-71577d9c-9ff3-4ea6-a3b6-18be9b824729'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.352729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] The token cache is being invalidate because project e549d0bfa74f4fbebc107476b3927ec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:12.353017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.353677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.355287 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1134/2260] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:12 2022] DELETE /identity/v3/projects/e549d0bfa74f4fbebc107476b3927ec8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:12.359529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07876a52-af3f-4831-8de9-d8a5de8ea393 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:12.369504 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07876a52-af3f-4831-8de9-d8a5de8ea393 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:12.431913 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870cc39a-915d-43bc-b8a5-48cac32b2a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:12.432544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cc39a-915d-43bc-b8a5-48cac32b2a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:12.432862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cc39a-915d-43bc-b8a5-48cac32b2a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:12.433123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870cc39a-915d-43bc-b8a5-48cac32b2a69 admin admin] PATH_INFO: `/v3/users/e748eebed0b54b2d8eaa4cb0bfd7f30a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:12.435004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cc39a-915d-43bc-b8a5-48cac32b2a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e748eebed0b54b2d8eaa4cb0bfd7f30a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:12.436454 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870cc39a-915d-43bc-b8a5-48cac32b2a69 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:12.485159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-870cc39a-915d-43bc-b8a5-48cac32b2a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e748eebed0b54b2d8eaa4cb0bfd7f30a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.492518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-870cc39a-915d-43bc-b8a5-48cac32b2a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e748eebed0b54b2d8eaa4cb0bfd7f30a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.506062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-870cc39a-915d-43bc-b8a5-48cac32b2a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e748eebed0b54b2d8eaa4cb0bfd7f30a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.516220 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-870cc39a-915d-43bc-b8a5-48cac32b2a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e748eebed0b54b2d8eaa4cb0bfd7f30a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.516902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-870cc39a-915d-43bc-b8a5-48cac32b2a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e748eebed0b54b2d8eaa4cb0bfd7f30a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.523076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1127/2261] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:12 2022] DELETE /identity/v3/users/e748eebed0b54b2d8eaa4cb0bfd7f30a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:12.813212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:12.814705 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:12.828118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71577d9c-9ff3-4ea6-a3b6-18be9b824729 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:12.896090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeed1be-8129-4e11-bf00-2c8c596686c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:12.896772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeed1be-8129-4e11-bf00-2c8c596686c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:12.897048 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeed1be-8129-4e11-bf00-2c8c596686c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:12.897362 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeed1be-8129-4e11-bf00-2c8c596686c5 admin admin] PATH_INFO: `/v3/projects/63cf9383f04d4e8994f1aa38f7bf2f1f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:12.898547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeed1be-8129-4e11-bf00-2c8c596686c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63cf9383f04d4e8994f1aa38f7bf2f1f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:12.900428 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeed1be-8129-4e11-bf00-2c8c596686c5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:12.949178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:12.972314 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3eeed1be-8129-4e11-bf00-2c8c596686c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63cf9383f04d4e8994f1aa38f7bf2f1f', 'request_id': 'req-3eeed1be-8129-4e11-bf00-2c8c596686c5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.981652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3eeed1be-8129-4e11-bf00-2c8c596686c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63cf9383f04d4e8994f1aa38f7bf2f1f', 'request_id': 'req-3eeed1be-8129-4e11-bf00-2c8c596686c5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:12.988772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3eeed1be-8129-4e11-bf00-2c8c596686c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63cf9383f04d4e8994f1aa38f7bf2f1f', 'request_id': 'req-3eeed1be-8129-4e11-bf00-2c8c596686c5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:13.000393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3eeed1be-8129-4e11-bf00-2c8c596686c5 admin admin] The token cache is being invalidate because project 63cf9383f04d4e8994f1aa38f7bf2f1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:13.000710 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3eeed1be-8129-4e11-bf00-2c8c596686c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:13.001352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3eeed1be-8129-4e11-bf00-2c8c596686c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:13.002883 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1135/2262] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:12 2022] DELETE /identity/v3/projects/63cf9383f04d4e8994f1aa38f7bf2f1f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:13.008041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edcedc-dac1-4224-aa0d-987fe57bf2c0 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.008523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edcedc-dac1-4224-aa0d-987fe57bf2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.008661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77edcedc-dac1-4224-aa0d-987fe57bf2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.017564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:13.029748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-77edcedc-dac1-4224-aa0d-987fe57bf2c0 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:13.030941 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-77edcedc-dac1-4224-aa0d-987fe57bf2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:13.095524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1128/2263] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:13.103405 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724ddcf-a29d-4f00-a7d2-b3e63e93b92a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.103720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724ddcf-a29d-4f00-a7d2-b3e63e93b92a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.103990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724ddcf-a29d-4f00-a7d2-b3e63e93b92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.124024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c724ddcf-a29d-4f00-a7d2-b3e63e93b92a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:13.125245 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c724ddcf-a29d-4f00-a7d2-b3e63e93b92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:13.133210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1136/2264] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:13.138122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edcedc-dac1-4224-aa0d-987fe57bf2c0 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.153202 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77edcedc-dac1-4224-aa0d-987fe57bf2c0 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.161809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd37f3-3218-4a3f-873d-53f4a37e63eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.162704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd37f3-3218-4a3f-873d-53f4a37e63eb admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.163019 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd37f3-3218-4a3f-873d-53f4a37e63eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.163394 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd37f3-3218-4a3f-873d-53f4a37e63eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.164397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd37f3-3218-4a3f-873d-53f4a37e63eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:13.164891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd37f3-3218-4a3f-873d-53f4a37e63eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.167394 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd37f3-3218-4a3f-873d-53f4a37e63eb admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.179576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1129/2265] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:13.184197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724ddcf-a29d-4f00-a7d2-b3e63e93b92a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.193828 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724ddcf-a29d-4f00-a7d2-b3e63e93b92a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.199540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322c1c8-6cba-44f3-91ac-7a59821e0269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.200139 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322c1c8-6cba-44f3-91ac-7a59821e0269 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.200410 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322c1c8-6cba-44f3-91ac-7a59821e0269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.200709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322c1c8-6cba-44f3-91ac-7a59821e0269 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.201345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322c1c8-6cba-44f3-91ac-7a59821e0269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.203236 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322c1c8-6cba-44f3-91ac-7a59821e0269 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.234248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1137/2266] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:13 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:13.238425 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd37f3-3218-4a3f-873d-53f4a37e63eb admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.250042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd37f3-3218-4a3f-873d-53f4a37e63eb admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.316813 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b37ad-c519-4f53-80ec-fbc9cec9bb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.317421 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b37ad-c519-4f53-80ec-fbc9cec9bb0c admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.317703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b37ad-c519-4f53-80ec-fbc9cec9bb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.317973 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b37ad-c519-4f53-80ec-fbc9cec9bb0c admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.318609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b37ad-c519-4f53-80ec-fbc9cec9bb0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.320529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b37ad-c519-4f53-80ec-fbc9cec9bb0c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.345836 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1130/2267] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:13 2022] POST /identity/v3/users => generated 284 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:13.350183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322c1c8-6cba-44f3-91ac-7a59821e0269 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.361504 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322c1c8-6cba-44f3-91ac-7a59821e0269 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.367205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d0fda-50cc-4292-8076-8e3a9407f64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.368001 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d0fda-50cc-4292-8076-8e3a9407f64b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.368281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d0fda-50cc-4292-8076-8e3a9407f64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.368550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687d0fda-50cc-4292-8076-8e3a9407f64b admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.369248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d0fda-50cc-4292-8076-8e3a9407f64b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.371052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687d0fda-50cc-4292-8076-8e3a9407f64b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.381177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1138/2268] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:13.385956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b37ad-c519-4f53-80ec-fbc9cec9bb0c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.400864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b37ad-c519-4f53-80ec-fbc9cec9bb0c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.406678 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b53529-c763-4b6b-bc62-cfede1616bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.407280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b53529-c763-4b6b-bc62-cfede1616bdf admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.407554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b53529-c763-4b6b-bc62-cfede1616bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.407820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b53529-c763-4b6b-bc62-cfede1616bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.408463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b53529-c763-4b6b-bc62-cfede1616bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.410097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b53529-c763-4b6b-bc62-cfede1616bdf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.422145 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1131/2269] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:13.426480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d0fda-50cc-4292-8076-8e3a9407f64b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.436654 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687d0fda-50cc-4292-8076-8e3a9407f64b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.441778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91893b7-ad26-4a5c-8b02-e584ee1a5cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.442427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91893b7-ad26-4a5c-8b02-e584ee1a5cca admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.442701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91893b7-ad26-4a5c-8b02-e584ee1a5cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.442972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91893b7-ad26-4a5c-8b02-e584ee1a5cca admin admin] PATH_INFO: `/v3/projects/c9527c36bf54477687e85805d2d0ab55/users/4bf077a2d23946979f35b8c295f0640b/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.458094 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91893b7-ad26-4a5c-8b02-e584ee1a5cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9527c36bf54477687e85805d2d0ab55, user_id=4bf077a2d23946979f35b8c295f0640b, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.460752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91893b7-ad26-4a5c-8b02-e584ee1a5cca admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.476357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1139/2270] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:13 2022] PUT /identity/v3/projects/c9527c36bf54477687e85805d2d0ab55/users/4bf077a2d23946979f35b8c295f0640b/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:13.487950 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c86f4e-14d3-4ab2-9646-31d17fb833be None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.488257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c86f4e-14d3-4ab2-9646-31d17fb833be None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.488580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c86f4e-14d3-4ab2-9646-31d17fb833be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.520648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-46c86f4e-14d3-4ab2-9646-31d17fb833be None None] MFA Rules not processed for user `4bf077a2d23946979f35b8c295f0640b`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:13.521778 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-46c86f4e-14d3-4ab2-9646-31d17fb833be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:13.591885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:13.595858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1132/2271] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:13.600716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91893b7-ad26-4a5c-8b02-e584ee1a5cca admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.672734 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91893b7-ad26-4a5c-8b02-e584ee1a5cca admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.678337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c8dad-0e76-4e55-8499-4ad022d17af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.678998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c8dad-0e76-4e55-8499-4ad022d17af0 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.678998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c8dad-0e76-4e55-8499-4ad022d17af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.678998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c8dad-0e76-4e55-8499-4ad022d17af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.680237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c8dad-0e76-4e55-8499-4ad022d17af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.682339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c8dad-0e76-4e55-8499-4ad022d17af0 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.718354 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1140/2272] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:13 2022] POST /identity/v3/projects => generated 363 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:13.722927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c86f4e-14d3-4ab2-9646-31d17fb833be None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.736167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c86f4e-14d3-4ab2-9646-31d17fb833be None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.802279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470054d-0c08-42c6-89e2-08a3c955c589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.802916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470054d-0c08-42c6-89e2-08a3c955c589 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.803196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470054d-0c08-42c6-89e2-08a3c955c589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.803509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470054d-0c08-42c6-89e2-08a3c955c589 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.804265 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470054d-0c08-42c6-89e2-08a3c955c589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.806150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470054d-0c08-42c6-89e2-08a3c955c589 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.838434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1133/2273] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:13 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:13.842590 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c8dad-0e76-4e55-8499-4ad022d17af0 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.853845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c8dad-0e76-4e55-8499-4ad022d17af0 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.860662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c03fd-4fc4-453f-a996-14127aaed2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.861353 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c03fd-4fc4-453f-a996-14127aaed2cc admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.861691 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c03fd-4fc4-453f-a996-14127aaed2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.862023 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c03fd-4fc4-453f-a996-14127aaed2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.862754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c03fd-4fc4-453f-a996-14127aaed2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.864436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c03fd-4fc4-453f-a996-14127aaed2cc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.879722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1141/2274] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:13.880571 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470054d-0c08-42c6-89e2-08a3c955c589 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.891147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470054d-0c08-42c6-89e2-08a3c955c589 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.895908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ac89a-d044-4ab3-ab70-bc5a873e4e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:13.896626 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ac89a-d044-4ab3-ab70-bc5a873e4e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:13.896898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ac89a-d044-4ab3-ab70-bc5a873e4e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:13.897167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ac89a-d044-4ab3-ab70-bc5a873e4e3e admin admin] PATH_INFO: `/v3/projects/2f5c9789a2494935bc99f2a82ba84efa/users/b9ae6e98c8fd42d0a3eac80e4eca8b82/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:13.913767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ac89a-d044-4ab3-ab70-bc5a873e4e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f5c9789a2494935bc99f2a82ba84efa, user_id=b9ae6e98c8fd42d0a3eac80e4eca8b82, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:13.916496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ac89a-d044-4ab3-ab70-bc5a873e4e3e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:13.932180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1134/2275] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:13 2022] PUT /identity/v3/projects/2f5c9789a2494935bc99f2a82ba84efa/users/b9ae6e98c8fd42d0a3eac80e4eca8b82/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:13.936452 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c03fd-4fc4-453f-a996-14127aaed2cc admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:13.946753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c03fd-4fc4-453f-a996-14127aaed2cc admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:13.969054 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:14.014921 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112dbbb5-7eb2-4053-b47c-283a5cf0bf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:14.015556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dbbb5-7eb2-4053-b47c-283a5cf0bf39 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:14.015974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dbbb5-7eb2-4053-b47c-283a5cf0bf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:14.016250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112dbbb5-7eb2-4053-b47c-283a5cf0bf39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:14.017016 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112dbbb5-7eb2-4053-b47c-283a5cf0bf39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:14.018648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112dbbb5-7eb2-4053-b47c-283a5cf0bf39 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:14.031441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1142/2276] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:14.033687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ac89a-d044-4ab3-ab70-bc5a873e4e3e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:14.046177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ac89a-d044-4ab3-ab70-bc5a873e4e3e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:14.051978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf932fe-c74c-4601-a7ec-507f6dba6d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:14.052616 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf932fe-c74c-4601-a7ec-507f6dba6d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:14.052901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf932fe-c74c-4601-a7ec-507f6dba6d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:14.053322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf932fe-c74c-4601-a7ec-507f6dba6d45 admin admin] PATH_INFO: `/v3/projects/2f5c9789a2494935bc99f2a82ba84efa/users/b9ae6e98c8fd42d0a3eac80e4eca8b82/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:14.055367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf932fe-c74c-4601-a7ec-507f6dba6d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f5c9789a2494935bc99f2a82ba84efa, user_id=b9ae6e98c8fd42d0a3eac80e4eca8b82, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:14.058913 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf932fe-c74c-4601-a7ec-507f6dba6d45 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:14.084822 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1135/2277] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:14 2022] PUT /identity/v3/projects/2f5c9789a2494935bc99f2a82ba84efa/users/b9ae6e98c8fd42d0a3eac80e4eca8b82/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:14.090039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa5e68-3956-4b70-83b2-3976a0e60a07 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:14.090441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa5e68-3956-4b70-83b2-3976a0e60a07 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:14.090786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa5e68-3956-4b70-83b2-3976a0e60a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:14.127816 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-76fa5e68-3956-4b70-83b2-3976a0e60a07 None None] MFA Rules not processed for user `b9ae6e98c8fd42d0a3eac80e4eca8b82`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:14.129447 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-76fa5e68-3956-4b70-83b2-3976a0e60a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:14.202868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1143/2278] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:14.214673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf932fe-c74c-4601-a7ec-507f6dba6d45 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:14.216757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9bf932fe-c74c-4601-a7ec-507f6dba6d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:14.277208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf932fe-c74c-4601-a7ec-507f6dba6d45 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:14.281791 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbb0cb-b85e-4b93-beed-3f9c52a4a0a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:14.282774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbb0cb-b85e-4b93-beed-3f9c52a4a0a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:14.282774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbb0cb-b85e-4b93-beed-3f9c52a4a0a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:14.283012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbb0cb-b85e-4b93-beed-3f9c52a4a0a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:14.295944 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbb0cb-b85e-4b93-beed-3f9c52a4a0a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:14.297580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbb0cb-b85e-4b93-beed-3f9c52a4a0a8 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:14.307403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1136/2279] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:14.405543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa5e68-3956-4b70-83b2-3976a0e60a07 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:14.416511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa5e68-3956-4b70-83b2-3976a0e60a07 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:14.421506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a55c190-e892-4245-93ab-b5bbed58e398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:14.422251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55c190-e892-4245-93ab-b5bbed58e398 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:14.422632 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55c190-e892-4245-93ab-b5bbed58e398 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:14.423173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a55c190-e892-4245-93ab-b5bbed58e398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:14.492351 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55c190-e892-4245-93ab-b5bbed58e398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:14.493849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a55c190-e892-4245-93ab-b5bbed58e398 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:14.515453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1144/2280] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:15.168033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbb0cb-b85e-4b93-beed-3f9c52a4a0a8 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:15.170694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-14fbb0cb-b85e-4b93-beed-3f9c52a4a0a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:15.235976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:15.241872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbb0cb-b85e-4b93-beed-3f9c52a4a0a8 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:15.247612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:15.248422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:15.248741 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:15.249443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] PATH_INFO: `/v3/users/55d8a79666da49f0998a9e0eab779aa2` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:15.250855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55d8a79666da49f0998a9e0eab779aa2)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:15.252796 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:15.304313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55d8a79666da49f0998a9e0eab779aa2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.312131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55d8a79666da49f0998a9e0eab779aa2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.325035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55d8a79666da49f0998a9e0eab779aa2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.334463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55d8a79666da49f0998a9e0eab779aa2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.335110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55d8a79666da49f0998a9e0eab779aa2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.341767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1137/2281] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:15 2022] DELETE /identity/v3/users/55d8a79666da49f0998a9e0eab779aa2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:15.346229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e8071-8387-451e-b1ff-21a21ad24459 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:15.346788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e8071-8387-451e-b1ff-21a21ad24459 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:15.346788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e8071-8387-451e-b1ff-21a21ad24459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:15.367607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d25e8071-8387-451e-b1ff-21a21ad24459 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:15.368776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d25e8071-8387-451e-b1ff-21a21ad24459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:15.409846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:15.429250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1145/2282] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:15.439151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:15.440783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:15.502065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2784b12-0baa-4554-a710-0b829d0ede64 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:15.506181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c5a8b-e216-4135-88b7-5ec98249a917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:15.506795 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c5a8b-e216-4135-88b7-5ec98249a917 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:15.507071 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c5a8b-e216-4135-88b7-5ec98249a917 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:15.507341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c5a8b-e216-4135-88b7-5ec98249a917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:15.516960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c5a8b-e216-4135-88b7-5ec98249a917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:15.518383 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c5a8b-e216-4135-88b7-5ec98249a917 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:15.528850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1138/2283] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:15.812684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e8071-8387-451e-b1ff-21a21ad24459 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:15.813928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d25e8071-8387-451e-b1ff-21a21ad24459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:15.834089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e8071-8387-451e-b1ff-21a21ad24459 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:15.841425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de280711-0e57-48a5-8047-dab724f85ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:15.842057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de280711-0e57-48a5-8047-dab724f85ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:15.842330 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de280711-0e57-48a5-8047-dab724f85ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:15.842598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de280711-0e57-48a5-8047-dab724f85ca5 admin admin] PATH_INFO: `/v3/projects/17ab30d675e244559a0cb2f7c87a4f5b` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:15.843855 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de280711-0e57-48a5-8047-dab724f85ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ab30d675e244559a0cb2f7c87a4f5b)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:15.845875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de280711-0e57-48a5-8047-dab724f85ca5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:15.915587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-de280711-0e57-48a5-8047-dab724f85ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ab30d675e244559a0cb2f7c87a4f5b', 'request_id': 'req-de280711-0e57-48a5-8047-dab724f85ca5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.936920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-de280711-0e57-48a5-8047-dab724f85ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ab30d675e244559a0cb2f7c87a4f5b', 'request_id': 'req-de280711-0e57-48a5-8047-dab724f85ca5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.945437 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-de280711-0e57-48a5-8047-dab724f85ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ab30d675e244559a0cb2f7c87a4f5b', 'request_id': 'req-de280711-0e57-48a5-8047-dab724f85ca5'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.975299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-de280711-0e57-48a5-8047-dab724f85ca5 admin admin] The token cache is being invalidate because project 17ab30d675e244559a0cb2f7c87a4f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:15.975741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-de280711-0e57-48a5-8047-dab724f85ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.976471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-de280711-0e57-48a5-8047-dab724f85ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:15.977992 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1146/2284] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:15 2022] DELETE /identity/v3/projects/17ab30d675e244559a0cb2f7c87a4f5b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:15.983171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d50693-5996-4ced-851b-651405961e2c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:15.983448 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d50693-5996-4ced-851b-651405961e2c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:15.983716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d50693-5996-4ced-851b-651405961e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.004442 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-70d50693-5996-4ced-851b-651405961e2c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:16.005714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-70d50693-5996-4ced-851b-651405961e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:16.080066 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1139/2285] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:16.088182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57531cdd-17a6-4712-a070-6cda42a78621 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.088462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57531cdd-17a6-4712-a070-6cda42a78621 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.088764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57531cdd-17a6-4712-a070-6cda42a78621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.111569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-57531cdd-17a6-4712-a070-6cda42a78621 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:16.112127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-57531cdd-17a6-4712-a070-6cda42a78621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:16.120932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1147/2286] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:16.125489 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d50693-5996-4ced-851b-651405961e2c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.138216 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d50693-5996-4ced-851b-651405961e2c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.159144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de544fa-0a22-4004-952e-2b1ef24c1138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.160989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de544fa-0a22-4004-952e-2b1ef24c1138 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.161468 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de544fa-0a22-4004-952e-2b1ef24c1138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.161700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de544fa-0a22-4004-952e-2b1ef24c1138 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.163567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de544fa-0a22-4004-952e-2b1ef24c1138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:16.163873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de544fa-0a22-4004-952e-2b1ef24c1138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.169503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de544fa-0a22-4004-952e-2b1ef24c1138 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.193640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1140/2287] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:21:16.195574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57531cdd-17a6-4712-a070-6cda42a78621 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.207492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57531cdd-17a6-4712-a070-6cda42a78621 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.213186 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8bed7-5efa-4546-b1a4-9a0c94fa70cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.213910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8bed7-5efa-4546-b1a4-9a0c94fa70cb admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.214430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8bed7-5efa-4546-b1a4-9a0c94fa70cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.214958 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8bed7-5efa-4546-b1a4-9a0c94fa70cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.215716 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8bed7-5efa-4546-b1a4-9a0c94fa70cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.218569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8bed7-5efa-4546-b1a4-9a0c94fa70cb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.265942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1148/2288] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:16 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:16.270773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de544fa-0a22-4004-952e-2b1ef24c1138 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.278592 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:16.286936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de544fa-0a22-4004-952e-2b1ef24c1138 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.384644 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabe47b-bb60-48e6-8898-c75e65a8d626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.385374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabe47b-bb60-48e6-8898-c75e65a8d626 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.385677 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabe47b-bb60-48e6-8898-c75e65a8d626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.385948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabe47b-bb60-48e6-8898-c75e65a8d626 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.387882 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabe47b-bb60-48e6-8898-c75e65a8d626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.391005 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabe47b-bb60-48e6-8898-c75e65a8d626 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.452196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8bed7-5efa-4546-b1a4-9a0c94fa70cb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.456745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1141/2289] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:16 2022] POST /identity/v3/users => generated 282 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:16.469480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8bed7-5efa-4546-b1a4-9a0c94fa70cb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.477698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d95e38-31a2-42e4-b43d-de7406edf3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.478422 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d95e38-31a2-42e4-b43d-de7406edf3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.478698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d95e38-31a2-42e4-b43d-de7406edf3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.478971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d95e38-31a2-42e4-b43d-de7406edf3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.479661 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d95e38-31a2-42e4-b43d-de7406edf3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.481535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d95e38-31a2-42e4-b43d-de7406edf3a3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.504126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabe47b-bb60-48e6-8898-c75e65a8d626 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.505718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1149/2290] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:16.516766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabe47b-bb60-48e6-8898-c75e65a8d626 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.521472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a36b8-85bd-4489-8d57-39888f4da1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.522090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a36b8-85bd-4489-8d57-39888f4da1dc admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.522365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a36b8-85bd-4489-8d57-39888f4da1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.522630 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a36b8-85bd-4489-8d57-39888f4da1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.524109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a36b8-85bd-4489-8d57-39888f4da1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.526217 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a36b8-85bd-4489-8d57-39888f4da1dc admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.540551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1142/2291] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:16.545222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d95e38-31a2-42e4-b43d-de7406edf3a3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.558247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d95e38-31a2-42e4-b43d-de7406edf3a3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.563150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd16d7c-1ed5-4cce-a76a-816f5a96faea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.563818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd16d7c-1ed5-4cce-a76a-816f5a96faea admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.564091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd16d7c-1ed5-4cce-a76a-816f5a96faea admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.564355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd16d7c-1ed5-4cce-a76a-816f5a96faea admin admin] PATH_INFO: `/v3/projects/6f972d1448134d4a8a1b68a34e40f643/users/78a175a45c6441be860bfbbb7599efda/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.582471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd16d7c-1ed5-4cce-a76a-816f5a96faea admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f972d1448134d4a8a1b68a34e40f643, user_id=78a175a45c6441be860bfbbb7599efda, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.585196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd16d7c-1ed5-4cce-a76a-816f5a96faea admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.604524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1150/2292] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:16 2022] PUT /identity/v3/projects/6f972d1448134d4a8a1b68a34e40f643/users/78a175a45c6441be860bfbbb7599efda/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:16.616429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe28316-f31b-49e6-9647-f59ecc828d08 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.616750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe28316-f31b-49e6-9647-f59ecc828d08 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.617026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe28316-f31b-49e6-9647-f59ecc828d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.645436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-dfe28316-f31b-49e6-9647-f59ecc828d08 None None] MFA Rules not processed for user `78a175a45c6441be860bfbbb7599efda`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:16.646664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dfe28316-f31b-49e6-9647-f59ecc828d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:16.712500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1143/2293] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:16.724008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd16d7c-1ed5-4cce-a76a-816f5a96faea admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:16.725718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-cfd16d7c-1ed5-4cce-a76a-816f5a96faea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:16.844225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd16d7c-1ed5-4cce-a76a-816f5a96faea admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:16.849112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103cdec-98e5-4d43-a0bc-4b8f5e7c5c55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:16.850052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103cdec-98e5-4d43-a0bc-4b8f5e7c5c55 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:16.850542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103cdec-98e5-4d43-a0bc-4b8f5e7c5c55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:16.851547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103cdec-98e5-4d43-a0bc-4b8f5e7c5c55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:16.866687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103cdec-98e5-4d43-a0bc-4b8f5e7c5c55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:16.868581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103cdec-98e5-4d43-a0bc-4b8f5e7c5c55 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:16.896536 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1151/2294] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:19.066262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe28316-f31b-49e6-9647-f59ecc828d08 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:19.068332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dfe28316-f31b-49e6-9647-f59ecc828d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:19.141790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe28316-f31b-49e6-9647-f59ecc828d08 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:19.146919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:19.147657 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.147926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.148188 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] PATH_INFO: `/v3/users/8733b36fe7e34ec2b88031ac8752ca8a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:19.149340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8733b36fe7e34ec2b88031ac8752ca8a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:19.151309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:19.165741 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:19.203187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8733b36fe7e34ec2b88031ac8752ca8a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.210868 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8733b36fe7e34ec2b88031ac8752ca8a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.223002 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8733b36fe7e34ec2b88031ac8752ca8a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.231726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8733b36fe7e34ec2b88031ac8752ca8a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.232345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8733b36fe7e34ec2b88031ac8752ca8a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.239448 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1144/2295] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:19 2022] DELETE /identity/v3/users/8733b36fe7e34ec2b88031ac8752ca8a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:19.244610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92394a3-71e6-4e23-8e35-ab938e205090 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.245201 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92394a3-71e6-4e23-8e35-ab938e205090 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.245201 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92394a3-71e6-4e23-8e35-ab938e205090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:19.258785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:19.265383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c92394a3-71e6-4e23-8e35-ab938e205090 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:19.267430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c92394a3-71e6-4e23-8e35-ab938e205090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:19.329983 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1152/2296] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:19.340971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:19.342923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:19.405646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56b644-410a-4d3c-a260-6d62eb50a222 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:19.409442 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a0568-3dd3-48c2-9649-85da3a9d4a87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:19.410054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a0568-3dd3-48c2-9649-85da3a9d4a87 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.410324 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a0568-3dd3-48c2-9649-85da3a9d4a87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.410585 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a0568-3dd3-48c2-9649-85da3a9d4a87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:19.420834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a0568-3dd3-48c2-9649-85da3a9d4a87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:19.422252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a0568-3dd3-48c2-9649-85da3a9d4a87 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:19.432112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1145/2297] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:19.728974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92394a3-71e6-4e23-8e35-ab938e205090 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:19.730861 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c92394a3-71e6-4e23-8e35-ab938e205090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:19.744889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92394a3-71e6-4e23-8e35-ab938e205090 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:19.750445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02074a3-f079-4515-9131-ad7c610b7339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:19.753361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02074a3-f079-4515-9131-ad7c610b7339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.753732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02074a3-f079-4515-9131-ad7c610b7339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.754098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02074a3-f079-4515-9131-ad7c610b7339 admin admin] PATH_INFO: `/v3/projects/1d52492061fd4f37a6a3b538f720eeb7` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:19.755333 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02074a3-f079-4515-9131-ad7c610b7339 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d52492061fd4f37a6a3b538f720eeb7)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:19.757773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02074a3-f079-4515-9131-ad7c610b7339 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:19.809043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b02074a3-f079-4515-9131-ad7c610b7339 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d52492061fd4f37a6a3b538f720eeb7', 'request_id': 'req-b02074a3-f079-4515-9131-ad7c610b7339'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.817800 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b02074a3-f079-4515-9131-ad7c610b7339 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d52492061fd4f37a6a3b538f720eeb7', 'request_id': 'req-b02074a3-f079-4515-9131-ad7c610b7339'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.825106 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b02074a3-f079-4515-9131-ad7c610b7339 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d52492061fd4f37a6a3b538f720eeb7', 'request_id': 'req-b02074a3-f079-4515-9131-ad7c610b7339'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.836070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b02074a3-f079-4515-9131-ad7c610b7339 admin admin] The token cache is being invalidate because project 1d52492061fd4f37a6a3b538f720eeb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:19.836363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b02074a3-f079-4515-9131-ad7c610b7339 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.837028 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b02074a3-f079-4515-9131-ad7c610b7339 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:19.838576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1153/2298] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:19 2022] DELETE /identity/v3/projects/1d52492061fd4f37a6a3b538f720eeb7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:19.844535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c5f73-a977-491d-9f5c-75a7b25c0790 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.844863 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c5f73-a977-491d-9f5c-75a7b25c0790 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.845131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c5f73-a977-491d-9f5c-75a7b25c0790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:19.868082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-fb8c5f73-a977-491d-9f5c-75a7b25c0790 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:19.869297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fb8c5f73-a977-491d-9f5c-75a7b25c0790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:19.969104 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1146/2299] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:19.978516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439aad6-099e-4060-a3ad-d8c61ce9b627 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:19.978800 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439aad6-099e-4060-a3ad-d8c61ce9b627 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:19.979081 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439aad6-099e-4060-a3ad-d8c61ce9b627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.013954 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0439aad6-099e-4060-a3ad-d8c61ce9b627 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:20.015233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0439aad6-099e-4060-a3ad-d8c61ce9b627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:20.042316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1154/2300] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:20.046952 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c5f73-a977-491d-9f5c-75a7b25c0790 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:20.063888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c5f73-a977-491d-9f5c-75a7b25c0790 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:20.081180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241dc50d-934d-4a7d-ba2a-207474666db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:20.081819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dc50d-934d-4a7d-ba2a-207474666db3 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.084978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dc50d-934d-4a7d-ba2a-207474666db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.085254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dc50d-934d-4a7d-ba2a-207474666db3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.086586 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241dc50d-934d-4a7d-ba2a-207474666db3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:20.086898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241dc50d-934d-4a7d-ba2a-207474666db3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:20.088493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241dc50d-934d-4a7d-ba2a-207474666db3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:20.113050 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1147/2301] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:20.118184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439aad6-099e-4060-a3ad-d8c61ce9b627 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:20.128741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439aad6-099e-4060-a3ad-d8c61ce9b627 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:20.149726 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc66d7a-eda2-49b5-bab8-8dc1b2f157a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:20.150454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc66d7a-eda2-49b5-bab8-8dc1b2f157a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.150910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc66d7a-eda2-49b5-bab8-8dc1b2f157a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.151281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc66d7a-eda2-49b5-bab8-8dc1b2f157a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.152149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc66d7a-eda2-49b5-bab8-8dc1b2f157a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:20.154794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc66d7a-eda2-49b5-bab8-8dc1b2f157a6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:20.155075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241dc50d-934d-4a7d-ba2a-207474666db3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:20.157115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-241dc50d-934d-4a7d-ba2a-207474666db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:20.173621 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241dc50d-934d-4a7d-ba2a-207474666db3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:20.182559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144ef70-947d-4b8f-a1ba-13280935c14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:20.183466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144ef70-947d-4b8f-a1ba-13280935c14b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.184018 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144ef70-947d-4b8f-a1ba-13280935c14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.185127 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144ef70-947d-4b8f-a1ba-13280935c14b admin admin] PATH_INFO: `/v3/users/4bf077a2d23946979f35b8c295f0640b` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.186781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144ef70-947d-4b8f-a1ba-13280935c14b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bf077a2d23946979f35b8c295f0640b)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:20.198580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144ef70-947d-4b8f-a1ba-13280935c14b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:20.201924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1155/2302] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:20 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:20.216484 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc66d7a-eda2-49b5-bab8-8dc1b2f157a6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:20.267795 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:20.318386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc66d7a-eda2-49b5-bab8-8dc1b2f157a6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:20.327522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d959a5-7a9d-4d85-9153-6ff9ac974ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:20.328355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d959a5-7a9d-4d85-9153-6ff9ac974ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.328892 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d959a5-7a9d-4d85-9153-6ff9ac974ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.329305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d959a5-7a9d-4d85-9153-6ff9ac974ef3 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.330225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d959a5-7a9d-4d85-9153-6ff9ac974ef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:20.332967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d959a5-7a9d-4d85-9153-6ff9ac974ef3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:20.337398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8144ef70-947d-4b8f-a1ba-13280935c14b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bf077a2d23946979f35b8c295f0640b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:20.345180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:20.357060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8144ef70-947d-4b8f-a1ba-13280935c14b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bf077a2d23946979f35b8c295f0640b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:20.378945 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1156/2303] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:20 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:20.379831 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8144ef70-947d-4b8f-a1ba-13280935c14b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bf077a2d23946979f35b8c295f0640b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:20.382316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d959a5-7a9d-4d85-9153-6ff9ac974ef3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:20.393415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8144ef70-947d-4b8f-a1ba-13280935c14b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bf077a2d23946979f35b8c295f0640b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:20.395044 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-8144ef70-947d-4b8f-a1ba-13280935c14b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bf077a2d23946979f35b8c295f0640b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:20.405574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d959a5-7a9d-4d85-9153-6ff9ac974ef3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:20.407962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1148/2304] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:20 2022] DELETE /identity/v3/users/4bf077a2d23946979f35b8c295f0640b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:20.410175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d57a2-8900-473a-997f-93dbb7caea8b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.411748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d57a2-8900-473a-997f-93dbb7caea8b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.412294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d57a2-8900-473a-997f-93dbb7caea8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.447854 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0e4d57a2-8900-473a-997f-93dbb7caea8b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:20.451399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0e4d57a2-8900-473a-997f-93dbb7caea8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:20.510529 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0a237-b498-4a61-b155-df85c0537b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:20.511210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0a237-b498-4a61-b155-df85c0537b6c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.511540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0a237-b498-4a61-b155-df85c0537b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.511804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0a237-b498-4a61-b155-df85c0537b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.512454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0a237-b498-4a61-b155-df85c0537b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:20.514277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0a237-b498-4a61-b155-df85c0537b6c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:20.528107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1157/2305] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:20.529358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0a237-b498-4a61-b155-df85c0537b6c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:20.531356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3ca0a237-b498-4a61-b155-df85c0537b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:20.546877 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0a237-b498-4a61-b155-df85c0537b6c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:20.552819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1149/2306] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:20.554283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67c50d-57b1-49d9-9968-87b5e950942d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:20.561408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67c50d-57b1-49d9-9968-87b5e950942d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.561657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67c50d-57b1-49d9-9968-87b5e950942d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.561881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67c50d-57b1-49d9-9968-87b5e950942d admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.562515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67c50d-57b1-49d9-9968-87b5e950942d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:20.564160 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67c50d-57b1-49d9-9968-87b5e950942d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:20.565155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d57a2-8900-473a-997f-93dbb7caea8b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:20.567908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0e4d57a2-8900-473a-997f-93dbb7caea8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:20.579644 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1158/2307] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:20.585845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67c50d-57b1-49d9-9968-87b5e950942d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:20.601662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67c50d-57b1-49d9-9968-87b5e950942d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:20.608112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbb574-b92e-4432-80c2-7958fac901a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:20.608872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbb574-b92e-4432-80c2-7958fac901a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.609622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbb574-b92e-4432-80c2-7958fac901a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.610180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbb574-b92e-4432-80c2-7958fac901a5 admin admin] PATH_INFO: `/v3/projects/9ce3cc699fd54ce8af2a9864646b3481/users/fb36dfeeac224133a00ae8c8282538bd/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.628095 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbb574-b92e-4432-80c2-7958fac901a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce3cc699fd54ce8af2a9864646b3481, user_id=fb36dfeeac224133a00ae8c8282538bd, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:20.631420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fbb574-b92e-4432-80c2-7958fac901a5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:20.644688 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d57a2-8900-473a-997f-93dbb7caea8b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:20.653740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1159/2308] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:20 2022] PUT /identity/v3/projects/9ce3cc699fd54ce8af2a9864646b3481/users/fb36dfeeac224133a00ae8c8282538bd/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:20.665086 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905510f-27fc-4fac-a425-392703ee5f15 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.665364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905510f-27fc-4fac-a425-392703ee5f15 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.665649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905510f-27fc-4fac-a425-392703ee5f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.705598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7905510f-27fc-4fac-a425-392703ee5f15 None None] MFA Rules not processed for user `fb36dfeeac224133a00ae8c8282538bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:20.707118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7905510f-27fc-4fac-a425-392703ee5f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:20.723268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfbd86-9b37-4c72-8533-c84a4496e63b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:20.723950 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfbd86-9b37-4c72-8533-c84a4496e63b service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.724286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfbd86-9b37-4c72-8533-c84a4496e63b service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.724634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfbd86-9b37-4c72-8533-c84a4496e63b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.729487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:20.787285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1160/2309] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:20.789837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfbd86-9b37-4c72-8533-c84a4496e63b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:20.791264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfbd86-9b37-4c72-8533-c84a4496e63b service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:20.791989 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905510f-27fc-4fac-a425-392703ee5f15 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:20.801030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1150/2310] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:20.802445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905510f-27fc-4fac-a425-392703ee5f15 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:20.807122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b2648-0593-47e3-b759-47434e6681c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:20.807722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b2648-0593-47e3-b759-47434e6681c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.807998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b2648-0593-47e3-b759-47434e6681c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.808307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b2648-0593-47e3-b759-47434e6681c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.809261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b2648-0593-47e3-b759-47434e6681c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:20.811176 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b2648-0593-47e3-b759-47434e6681c1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:20.842141 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1161/2311] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:20 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:20.846068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfbd86-9b37-4c72-8533-c84a4496e63b service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:20.857615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfbd86-9b37-4c72-8533-c84a4496e63b service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:20.915965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441e70e-90c3-49ec-be9f-3b19f6b935ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:20.916582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441e70e-90c3-49ec-be9f-3b19f6b935ad admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.916877 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441e70e-90c3-49ec-be9f-3b19f6b935ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.917139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441e70e-90c3-49ec-be9f-3b19f6b935ad admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.917825 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441e70e-90c3-49ec-be9f-3b19f6b935ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:20.919719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441e70e-90c3-49ec-be9f-3b19f6b935ad admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:20.942642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1151/2312] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:20 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:20.946448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b2648-0593-47e3-b759-47434e6681c1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:20.960073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b2648-0593-47e3-b759-47434e6681c1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:20.965801 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863cd97-7bec-4ba5-808d-b2447055d4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:20.975277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863cd97-7bec-4ba5-808d-b2447055d4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:20.975467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863cd97-7bec-4ba5-808d-b2447055d4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:20.975732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863cd97-7bec-4ba5-808d-b2447055d4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:20.976445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863cd97-7bec-4ba5-808d-b2447055d4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:20.978159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863cd97-7bec-4ba5-808d-b2447055d4d3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:20.996039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1162/2313] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:21.000433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441e70e-90c3-49ec-be9f-3b19f6b935ad admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.011097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441e70e-90c3-49ec-be9f-3b19f6b935ad admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.016568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf67aa0-214c-4ac7-8397-5bade9ccca03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.017261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf67aa0-214c-4ac7-8397-5bade9ccca03 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.017528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf67aa0-214c-4ac7-8397-5bade9ccca03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.017791 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf67aa0-214c-4ac7-8397-5bade9ccca03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.018436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf67aa0-214c-4ac7-8397-5bade9ccca03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.020031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf67aa0-214c-4ac7-8397-5bade9ccca03 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.030850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1152/2314] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:21.035167 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863cd97-7bec-4ba5-808d-b2447055d4d3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.045932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863cd97-7bec-4ba5-808d-b2447055d4d3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.050988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77293a17-ce06-4b2d-bbc2-1e2102d87a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.051661 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77293a17-ce06-4b2d-bbc2-1e2102d87a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.051940 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77293a17-ce06-4b2d-bbc2-1e2102d87a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.052208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77293a17-ce06-4b2d-bbc2-1e2102d87a5e admin admin] PATH_INFO: `/v3/projects/3565f08c24dc465fb9a57b84705cbe2d/users/3bf7ad9e23f44116bb24ef9fb77409f9/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.056445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:21.069739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77293a17-ce06-4b2d-bbc2-1e2102d87a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3565f08c24dc465fb9a57b84705cbe2d, user_id=3bf7ad9e23f44116bb24ef9fb77409f9, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.072464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77293a17-ce06-4b2d-bbc2-1e2102d87a5e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.097484 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1163/2315] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:21 2022] PUT /identity/v3/projects/3565f08c24dc465fb9a57b84705cbe2d/users/3bf7ad9e23f44116bb24ef9fb77409f9/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:21.101419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb43b9c-a226-4c44-baab-d7f6aa8e0f89 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.101926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb43b9c-a226-4c44-baab-d7f6aa8e0f89 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.102143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb43b9c-a226-4c44-baab-d7f6aa8e0f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.114617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77293a17-ce06-4b2d-bbc2-1e2102d87a5e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.117523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-77293a17-ce06-4b2d-bbc2-1e2102d87a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:21.143101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-afb43b9c-a226-4c44-baab-d7f6aa8e0f89 None None] MFA Rules not processed for user `3bf7ad9e23f44116bb24ef9fb77409f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:21.144529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-afb43b9c-a226-4c44-baab-d7f6aa8e0f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:21.193970 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77293a17-ce06-4b2d-bbc2-1e2102d87a5e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.199230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.199903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.200179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.200651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] PATH_INFO: `/v3/projects/c9527c36bf54477687e85805d2d0ab55` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.201636 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9527c36bf54477687e85805d2d0ab55)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.203598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.220147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1153/2316] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:21.224330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb43b9c-a226-4c44-baab-d7f6aa8e0f89 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.236559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb43b9c-a226-4c44-baab-d7f6aa8e0f89 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.241823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c763a-22c6-4367-bca1-84b9cfe37e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.242435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c763a-22c6-4367-bca1-84b9cfe37e05 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.242869 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c763a-22c6-4367-bca1-84b9cfe37e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.243274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c763a-22c6-4367-bca1-84b9cfe37e05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.243765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c763a-22c6-4367-bca1-84b9cfe37e05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.246418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c763a-22c6-4367-bca1-84b9cfe37e05 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.252017 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:21.268277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9527c36bf54477687e85805d2d0ab55', 'request_id': 'req-2c8129b0-e613-4865-8afc-c6ca489f5555'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.277968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9527c36bf54477687e85805d2d0ab55', 'request_id': 'req-2c8129b0-e613-4865-8afc-c6ca489f5555'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.285450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9527c36bf54477687e85805d2d0ab55', 'request_id': 'req-2c8129b0-e613-4865-8afc-c6ca489f5555'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.285747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1154/2317] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:21 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:21.289766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c763a-22c6-4367-bca1-84b9cfe37e05 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.298788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] The token cache is being invalidate because project c9527c36bf54477687e85805d2d0ab55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:21.299138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.312618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.312618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1164/2318] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:21 2022] DELETE /identity/v3/projects/c9527c36bf54477687e85805d2d0ab55 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:21.312618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.312618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:21.361726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c763a-22c6-4367-bca1-84b9cfe37e05 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.361726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ba6f4-a593-4a0f-a753-771560b08ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.361726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ba6f4-a593-4a0f-a753-771560b08ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.361726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ba6f4-a593-4a0f-a753-771560b08ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.362272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ba6f4-a593-4a0f-a753-771560b08ca1 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.362272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ba6f4-a593-4a0f-a753-771560b08ca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.362702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ba6f4-a593-4a0f-a753-771560b08ca1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.379419 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8129b0-e613-4865-8afc-c6ca489f5555 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.384797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d95de5-64be-442e-a55a-29f834eb1951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.385430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d95de5-64be-442e-a55a-29f834eb1951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.385701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d95de5-64be-442e-a55a-29f834eb1951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.385969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d95de5-64be-442e-a55a-29f834eb1951 admin admin] PATH_INFO: `/v3/users/b9ae6e98c8fd42d0a3eac80e4eca8b82` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.386902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1155/2319] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:21 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:21.387475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d95de5-64be-442e-a55a-29f834eb1951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9ae6e98c8fd42d0a3eac80e4eca8b82)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.389429 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d95de5-64be-442e-a55a-29f834eb1951 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.391196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ba6f4-a593-4a0f-a753-771560b08ca1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.392895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7e8ba6f4-a593-4a0f-a753-771560b08ca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:21.405823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ba6f4-a593-4a0f-a753-771560b08ca1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.407833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:21.410588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59e978-48ad-42d3-a306-6031050b6704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.411214 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59e978-48ad-42d3-a306-6031050b6704 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.411507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59e978-48ad-42d3-a306-6031050b6704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.411753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59e978-48ad-42d3-a306-6031050b6704 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.412392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59e978-48ad-42d3-a306-6031050b6704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.414053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59e978-48ad-42d3-a306-6031050b6704 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.424564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1156/2320] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:21.428823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59e978-48ad-42d3-a306-6031050b6704 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.439016 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59e978-48ad-42d3-a306-6031050b6704 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.443246 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666e713-da7a-4c8b-bd9a-4688775218d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.443898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666e713-da7a-4c8b-bd9a-4688775218d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.444269 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666e713-da7a-4c8b-bd9a-4688775218d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.444446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666e713-da7a-4c8b-bd9a-4688775218d4 admin admin] PATH_INFO: `/v3/projects/f36e502ec40646c6a7741f7c49b54ed1/users/211fce92d7e243b293c2f7e3641593cb/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.449905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-49d95de5-64be-442e-a55a-29f834eb1951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9ae6e98c8fd42d0a3eac80e4eca8b82'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.450707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-49d95de5-64be-442e-a55a-29f834eb1951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9ae6e98c8fd42d0a3eac80e4eca8b82'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.460208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666e713-da7a-4c8b-bd9a-4688775218d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36e502ec40646c6a7741f7c49b54ed1, user_id=211fce92d7e243b293c2f7e3641593cb, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.461479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-49d95de5-64be-442e-a55a-29f834eb1951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9ae6e98c8fd42d0a3eac80e4eca8b82'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.463028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666e713-da7a-4c8b-bd9a-4688775218d4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.475276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-49d95de5-64be-442e-a55a-29f834eb1951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9ae6e98c8fd42d0a3eac80e4eca8b82'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.476107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-49d95de5-64be-442e-a55a-29f834eb1951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9ae6e98c8fd42d0a3eac80e4eca8b82'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.479589 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1157/2321] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:21 2022] PUT /identity/v3/projects/f36e502ec40646c6a7741f7c49b54ed1/users/211fce92d7e243b293c2f7e3641593cb/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:21.483511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666e713-da7a-4c8b-bd9a-4688775218d4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.485306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d666e713-da7a-4c8b-bd9a-4688775218d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:21.494614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1165/2322] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:21 2022] DELETE /identity/v3/users/b9ae6e98c8fd42d0a3eac80e4eca8b82 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:21.593748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666e713-da7a-4c8b-bd9a-4688775218d4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.649606 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:21.664718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd5045-a76d-4744-9055-6a763383c350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.665338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd5045-a76d-4744-9055-6a763383c350 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.665611 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd5045-a76d-4744-9055-6a763383c350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.665890 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd5045-a76d-4744-9055-6a763383c350 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.666567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd5045-a76d-4744-9055-6a763383c350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.668188 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd5045-a76d-4744-9055-6a763383c350 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.678514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1158/2323] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:21.682934 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d95de5-64be-442e-a55a-29f834eb1951 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.696142 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d95de5-64be-442e-a55a-29f834eb1951 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.701786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e666e-cd93-45d6-a499-c48b1e8ababf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.702506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e666e-cd93-45d6-a499-c48b1e8ababf admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.702777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e666e-cd93-45d6-a499-c48b1e8ababf admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.703042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e666e-cd93-45d6-a499-c48b1e8ababf admin admin] PATH_INFO: `/v3/projects/f36e502ec40646c6a7741f7c49b54ed1/users/211fce92d7e243b293c2f7e3641593cb/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.705109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e666e-cd93-45d6-a499-c48b1e8ababf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f36e502ec40646c6a7741f7c49b54ed1, user_id=211fce92d7e243b293c2f7e3641593cb, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.707817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e666e-cd93-45d6-a499-c48b1e8ababf admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.720210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1166/2324] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:21 2022] PUT /identity/v3/projects/f36e502ec40646c6a7741f7c49b54ed1/users/211fce92d7e243b293c2f7e3641593cb/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:21.731871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029452a-9770-45b5-a46b-926185192a9f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.732156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029452a-9770-45b5-a46b-926185192a9f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.732431 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029452a-9770-45b5-a46b-926185192a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.769924 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4029452a-9770-45b5-a46b-926185192a9f None None] MFA Rules not processed for user `211fce92d7e243b293c2f7e3641593cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:21.771618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4029452a-9770-45b5-a46b-926185192a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:21.853845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e666e-cd93-45d6-a499-c48b1e8ababf admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.856223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3c9e666e-cd93-45d6-a499-c48b1e8ababf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:21.857336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1159/2325] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:21.871006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029452a-9770-45b5-a46b-926185192a9f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:21.873114 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4029452a-9770-45b5-a46b-926185192a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:21.932206 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029452a-9770-45b5-a46b-926185192a9f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.934660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e666e-cd93-45d6-a499-c48b1e8ababf admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:21.936051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58efec-d4b1-45a9-b3e0-ac896fcb362b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.936695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58efec-d4b1-45a9-b3e0-ac896fcb362b service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.936969 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58efec-d4b1-45a9-b3e0-ac896fcb362b service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.937231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58efec-d4b1-45a9-b3e0-ac896fcb362b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.938978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7a58efec-d4b1-45a9-b3e0-ac896fcb362b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:21.939365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:21.939991 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:21.940289 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:21.940558 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1 admin admin] PATH_INFO: `/v3/projects/2f5c9789a2494935bc99f2a82ba84efa` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:21.941708 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f5c9789a2494935bc99f2a82ba84efa)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:21.943621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:21.993543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f5c9789a2494935bc99f2a82ba84efa', 'request_id': 'req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:21.998542 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58efec-d4b1-45a9-b3e0-ac896fcb362b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.000189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58efec-d4b1-45a9-b3e0-ac896fcb362b service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.002289 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f5c9789a2494935bc99f2a82ba84efa', 'request_id': 'req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:22.008897 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f5c9789a2494935bc99f2a82ba84efa', 'request_id': 'req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:22.010215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1160/2326] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:22.020262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1 admin admin] The token cache is being invalidate because project 2f5c9789a2494935bc99f2a82ba84efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:22.020487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:22.021246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6daa4d3b-cb9a-4a7d-98f5-e6080cc8a2d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:22.023181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1167/2327] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:21 2022] DELETE /identity/v3/projects/2f5c9789a2494935bc99f2a82ba84efa => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:22.028485 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69046e0f-1584-43b8-b6ad-2bcfea5c4589 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.028783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69046e0f-1584-43b8-b6ad-2bcfea5c4589 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.029054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69046e0f-1584-43b8-b6ad-2bcfea5c4589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.049029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-69046e0f-1584-43b8-b6ad-2bcfea5c4589 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:22.050235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-69046e0f-1584-43b8-b6ad-2bcfea5c4589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:22.071244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:22.114337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1161/2328] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:22.122894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb32f7-c2a4-4dc5-861e-a9d23cdc376b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.123218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb32f7-c2a4-4dc5-861e-a9d23cdc376b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.123510 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb32f7-c2a4-4dc5-861e-a9d23cdc376b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.145061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c5cb32f7-c2a4-4dc5-861e-a9d23cdc376b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:22.146339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c5cb32f7-c2a4-4dc5-861e-a9d23cdc376b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:22.155741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1168/2329] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:22.160847 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69046e0f-1584-43b8-b6ad-2bcfea5c4589 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.172006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69046e0f-1584-43b8-b6ad-2bcfea5c4589 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.176866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38410f1d-1c32-4e2e-8fc1-cc60d14d63ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.177457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38410f1d-1c32-4e2e-8fc1-cc60d14d63ec admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.177728 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38410f1d-1c32-4e2e-8fc1-cc60d14d63ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.177995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38410f1d-1c32-4e2e-8fc1-cc60d14d63ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.178642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38410f1d-1c32-4e2e-8fc1-cc60d14d63ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:22.178954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38410f1d-1c32-4e2e-8fc1-cc60d14d63ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.180530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38410f1d-1c32-4e2e-8fc1-cc60d14d63ec admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.193643 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1162/2330] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:21:22.198351 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb32f7-c2a4-4dc5-861e-a9d23cdc376b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.204724 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:22.208660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb32f7-c2a4-4dc5-861e-a9d23cdc376b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.215016 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc98c4-a252-4590-a7bb-3fb8a79b78a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.216136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc98c4-a252-4590-a7bb-3fb8a79b78a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.216497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc98c4-a252-4590-a7bb-3fb8a79b78a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.217454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc98c4-a252-4590-a7bb-3fb8a79b78a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.218395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc98c4-a252-4590-a7bb-3fb8a79b78a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.221431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc98c4-a252-4590-a7bb-3fb8a79b78a8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.255869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1169/2331] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:22 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:22.260039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38410f1d-1c32-4e2e-8fc1-cc60d14d63ec admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.272638 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38410f1d-1c32-4e2e-8fc1-cc60d14d63ec admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.347896 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983067d-d164-4dfa-ade8-c9c109b0ec11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.348549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983067d-d164-4dfa-ade8-c9c109b0ec11 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.348857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983067d-d164-4dfa-ade8-c9c109b0ec11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.349126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983067d-d164-4dfa-ade8-c9c109b0ec11 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.349799 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983067d-d164-4dfa-ade8-c9c109b0ec11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.351855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983067d-d164-4dfa-ade8-c9c109b0ec11 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.379905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1163/2332] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:22 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:22.384000 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc98c4-a252-4590-a7bb-3fb8a79b78a8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.394774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc98c4-a252-4590-a7bb-3fb8a79b78a8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.399441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fafb5-5989-418d-b0f9-ef09c9e06463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.400030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fafb5-5989-418d-b0f9-ef09c9e06463 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.400304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fafb5-5989-418d-b0f9-ef09c9e06463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.400568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fafb5-5989-418d-b0f9-ef09c9e06463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.401246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fafb5-5989-418d-b0f9-ef09c9e06463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.403068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fafb5-5989-418d-b0f9-ef09c9e06463 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.413135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1170/2333] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:22.417164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983067d-d164-4dfa-ade8-c9c109b0ec11 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.427380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983067d-d164-4dfa-ade8-c9c109b0ec11 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.432030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb83c9e-fcfc-424c-93c8-a4d43e09a970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.432660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb83c9e-fcfc-424c-93c8-a4d43e09a970 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.432937 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb83c9e-fcfc-424c-93c8-a4d43e09a970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.433274 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb83c9e-fcfc-424c-93c8-a4d43e09a970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.434103 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb83c9e-fcfc-424c-93c8-a4d43e09a970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.435703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb83c9e-fcfc-424c-93c8-a4d43e09a970 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.447129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1164/2334] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:22.451434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fafb5-5989-418d-b0f9-ef09c9e06463 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.462295 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fafb5-5989-418d-b0f9-ef09c9e06463 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.467489 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3394d3-0ac8-4512-bba1-7d0eb14f7dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.468274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3394d3-0ac8-4512-bba1-7d0eb14f7dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.468638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3394d3-0ac8-4512-bba1-7d0eb14f7dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.468980 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3394d3-0ac8-4512-bba1-7d0eb14f7dcb admin admin] PATH_INFO: `/v3/projects/1bfbc9562a084551859d5b64e042afb0/users/a478befa2bbc4f7fb7e52250705bbc15/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.483973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3394d3-0ac8-4512-bba1-7d0eb14f7dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bfbc9562a084551859d5b64e042afb0, user_id=a478befa2bbc4f7fb7e52250705bbc15, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.486646 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3394d3-0ac8-4512-bba1-7d0eb14f7dcb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.505743 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1171/2335] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:22 2022] PUT /identity/v3/projects/1bfbc9562a084551859d5b64e042afb0/users/a478befa2bbc4f7fb7e52250705bbc15/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:22.517014 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e29794-d7f8-4e2b-8512-d69ffcab1c48 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.517295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e29794-d7f8-4e2b-8512-d69ffcab1c48 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.517561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e29794-d7f8-4e2b-8512-d69ffcab1c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.547526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-12e29794-d7f8-4e2b-8512-d69ffcab1c48 None None] MFA Rules not processed for user `a478befa2bbc4f7fb7e52250705bbc15`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:22.549239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-12e29794-d7f8-4e2b-8512-d69ffcab1c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:22.614924 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1165/2336] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:22.619870 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3394d3-0ac8-4512-bba1-7d0eb14f7dcb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.681380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3394d3-0ac8-4512-bba1-7d0eb14f7dcb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.686507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce711d51-5956-4cd6-84f2-1a1348e44d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.687100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce711d51-5956-4cd6-84f2-1a1348e44d09 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.687369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce711d51-5956-4cd6-84f2-1a1348e44d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.687628 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce711d51-5956-4cd6-84f2-1a1348e44d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.688249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce711d51-5956-4cd6-84f2-1a1348e44d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.690162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce711d51-5956-4cd6-84f2-1a1348e44d09 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.717479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1172/2337] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:22 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:22.721810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e29794-d7f8-4e2b-8512-d69ffcab1c48 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.733372 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e29794-d7f8-4e2b-8512-d69ffcab1c48 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.750653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:22.793863 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec3e58-98f3-4285-b5ec-f4e31d665cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.794436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec3e58-98f3-4285-b5ec-f4e31d665cce admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.794686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec3e58-98f3-4285-b5ec-f4e31d665cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.794949 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec3e58-98f3-4285-b5ec-f4e31d665cce admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.795588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec3e58-98f3-4285-b5ec-f4e31d665cce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.797537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec3e58-98f3-4285-b5ec-f4e31d665cce admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.821417 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1166/2338] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:22 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:22.825457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce711d51-5956-4cd6-84f2-1a1348e44d09 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.835525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce711d51-5956-4cd6-84f2-1a1348e44d09 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.840450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eadd56-9a4f-4c58-a684-2e1ead26a77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.841070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eadd56-9a4f-4c58-a684-2e1ead26a77e admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.841343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eadd56-9a4f-4c58-a684-2e1ead26a77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.841603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eadd56-9a4f-4c58-a684-2e1ead26a77e admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.842239 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eadd56-9a4f-4c58-a684-2e1ead26a77e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.843820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eadd56-9a4f-4c58-a684-2e1ead26a77e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.854288 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1173/2339] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:22.859163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec3e58-98f3-4285-b5ec-f4e31d665cce admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.869644 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec3e58-98f3-4285-b5ec-f4e31d665cce admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.875169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785d3a0-5fb6-4f9a-9ab7-f14e8e39d659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.875835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785d3a0-5fb6-4f9a-9ab7-f14e8e39d659 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.876105 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785d3a0-5fb6-4f9a-9ab7-f14e8e39d659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.876372 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785d3a0-5fb6-4f9a-9ab7-f14e8e39d659 admin admin] PATH_INFO: `/v3/projects/ac2d218f7b3a47d6b96df5cd7871fb1b/users/1b9e930927c345698d5ce806ff3173a4/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.893075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785d3a0-5fb6-4f9a-9ab7-f14e8e39d659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac2d218f7b3a47d6b96df5cd7871fb1b, user_id=1b9e930927c345698d5ce806ff3173a4, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.895751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785d3a0-5fb6-4f9a-9ab7-f14e8e39d659 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:22.909934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1167/2340] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:22 2022] PUT /identity/v3/projects/ac2d218f7b3a47d6b96df5cd7871fb1b/users/1b9e930927c345698d5ce806ff3173a4/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:22.914032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eadd56-9a4f-4c58-a684-2e1ead26a77e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:22.924792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eadd56-9a4f-4c58-a684-2e1ead26a77e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:22.989304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ed10d-d69c-4731-b916-b9b5779e01b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:22.989944 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ed10d-d69c-4731-b916-b9b5779e01b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:22.990571 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ed10d-d69c-4731-b916-b9b5779e01b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:22.990831 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ed10d-d69c-4731-b916-b9b5779e01b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:22.991479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ed10d-d69c-4731-b916-b9b5779e01b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:22.993120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ed10d-d69c-4731-b916-b9b5779e01b6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:23.003924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1174/2341] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:23.009272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785d3a0-5fb6-4f9a-9ab7-f14e8e39d659 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:23.023977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785d3a0-5fb6-4f9a-9ab7-f14e8e39d659 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:23.030052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240008c-7ae5-4ccb-b7ac-624c22eba7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:23.030744 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240008c-7ae5-4ccb-b7ac-624c22eba7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:23.030997 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240008c-7ae5-4ccb-b7ac-624c22eba7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:23.031283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240008c-7ae5-4ccb-b7ac-624c22eba7f7 admin admin] PATH_INFO: `/v3/projects/ac2d218f7b3a47d6b96df5cd7871fb1b/users/1b9e930927c345698d5ce806ff3173a4/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:23.033426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240008c-7ae5-4ccb-b7ac-624c22eba7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac2d218f7b3a47d6b96df5cd7871fb1b, user_id=1b9e930927c345698d5ce806ff3173a4, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:23.036048 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240008c-7ae5-4ccb-b7ac-624c22eba7f7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:23.048476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1168/2342] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:23 2022] PUT /identity/v3/projects/ac2d218f7b3a47d6b96df5cd7871fb1b/users/1b9e930927c345698d5ce806ff3173a4/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:23.060362 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb034b-6f86-40dd-8db1-01c9a63e092f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:23.060937 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb034b-6f86-40dd-8db1-01c9a63e092f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:23.061437 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb034b-6f86-40dd-8db1-01c9a63e092f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:23.099702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-fdcb034b-6f86-40dd-8db1-01c9a63e092f None None] MFA Rules not processed for user `1b9e930927c345698d5ce806ff3173a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:23.101035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fdcb034b-6f86-40dd-8db1-01c9a63e092f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:23.106025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:23.175039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1175/2343] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:23.188238 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240008c-7ae5-4ccb-b7ac-624c22eba7f7 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:23.190293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a240008c-7ae5-4ccb-b7ac-624c22eba7f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:23.258236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240008c-7ae5-4ccb-b7ac-624c22eba7f7 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:23.261810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac0bf2-441a-4ed1-837d-3e1f24ccee7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:23.262604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac0bf2-441a-4ed1-837d-3e1f24ccee7b service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:23.262872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac0bf2-441a-4ed1-837d-3e1f24ccee7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:23.263133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac0bf2-441a-4ed1-837d-3e1f24ccee7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:23.272003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac0bf2-441a-4ed1-837d-3e1f24ccee7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:23.273444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac0bf2-441a-4ed1-837d-3e1f24ccee7b service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:23.283000 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1169/2344] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:23.685541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb034b-6f86-40dd-8db1-01c9a63e092f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:23.752704 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb034b-6f86-40dd-8db1-01c9a63e092f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:23.757225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:23.757861 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:23.758134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:23.758402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] PATH_INFO: `/v3/users/a478befa2bbc4f7fb7e52250705bbc15` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:23.759503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a478befa2bbc4f7fb7e52250705bbc15)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:23.761525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:23.815920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a478befa2bbc4f7fb7e52250705bbc15'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:23.816576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a478befa2bbc4f7fb7e52250705bbc15'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:23.823621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a478befa2bbc4f7fb7e52250705bbc15'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:23.835334 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a478befa2bbc4f7fb7e52250705bbc15'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:23.835984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a478befa2bbc4f7fb7e52250705bbc15'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:23.850095 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1176/2345] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:23 2022] DELETE /identity/v3/users/a478befa2bbc4f7fb7e52250705bbc15 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:23.854730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7daad-d0cd-4886-afdd-f82da7e3c1e6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:23.855035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7daad-d0cd-4886-afdd-f82da7e3c1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:23.855808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7daad-d0cd-4886-afdd-f82da7e3c1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:23.874682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a5a7daad-d0cd-4886-afdd-f82da7e3c1e6 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:23.875857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a5a7daad-d0cd-4886-afdd-f82da7e3c1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:23.939205 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1170/2346] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:23.949141 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:23.950740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:23.978115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:24.012630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b177d85-bfe9-4e82-9f39-3e448d2fbafb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:24.016551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce59824-2646-4a9f-b0ed-2a6425e164eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:24.017216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce59824-2646-4a9f-b0ed-2a6425e164eb service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:24.017506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce59824-2646-4a9f-b0ed-2a6425e164eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:24.017761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce59824-2646-4a9f-b0ed-2a6425e164eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:24.027444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce59824-2646-4a9f-b0ed-2a6425e164eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:24.029101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce59824-2646-4a9f-b0ed-2a6425e164eb service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:24.038758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1177/2347] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:24.319876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7daad-d0cd-4886-afdd-f82da7e3c1e6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:24.321412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a5a7daad-d0cd-4886-afdd-f82da7e3c1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:24.334113 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7daad-d0cd-4886-afdd-f82da7e3c1e6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:24.340122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:24.340864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:24.341209 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:24.341546 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] PATH_INFO: `/v3/projects/1bfbc9562a084551859d5b64e042afb0` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:24.342760 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bfbc9562a084551859d5b64e042afb0)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:24.344731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:24.356284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:24.394742 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bfbc9562a084551859d5b64e042afb0', 'request_id': 'req-575c27c6-18f9-46cb-b154-8c606d293491'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.400903 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bfbc9562a084551859d5b64e042afb0', 'request_id': 'req-575c27c6-18f9-46cb-b154-8c606d293491'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.406635 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bfbc9562a084551859d5b64e042afb0', 'request_id': 'req-575c27c6-18f9-46cb-b154-8c606d293491'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.420737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] The token cache is being invalidate because project 1bfbc9562a084551859d5b64e042afb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:24.421020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.421639 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.424483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1171/2348] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:24 2022] DELETE /identity/v3/projects/1bfbc9562a084551859d5b64e042afb0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:24.427760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce59824-2646-4a9f-b0ed-2a6425e164eb service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:24.438399 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce59824-2646-4a9f-b0ed-2a6425e164eb service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:24.512928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4e2ca-6191-46fd-8185-1648b33d5cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:24.513567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4e2ca-6191-46fd-8185-1648b33d5cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:24.513842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4e2ca-6191-46fd-8185-1648b33d5cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:24.514111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4e2ca-6191-46fd-8185-1648b33d5cf7 admin admin] PATH_INFO: `/v3/users/1b9e930927c345698d5ce806ff3173a4` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:24.515335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4e2ca-6191-46fd-8185-1648b33d5cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9e930927c345698d5ce806ff3173a4)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:24.517301 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4e2ca-6191-46fd-8185-1648b33d5cf7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:24.570926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ff4e2ca-6191-46fd-8185-1648b33d5cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9e930927c345698d5ce806ff3173a4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.571600 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ff4e2ca-6191-46fd-8185-1648b33d5cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9e930927c345698d5ce806ff3173a4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.578959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ff4e2ca-6191-46fd-8185-1648b33d5cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9e930927c345698d5ce806ff3173a4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.591497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ff4e2ca-6191-46fd-8185-1648b33d5cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9e930927c345698d5ce806ff3173a4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.592281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7ff4e2ca-6191-46fd-8185-1648b33d5cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9e930927c345698d5ce806ff3173a4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:24.606847 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1178/2349] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:24 2022] DELETE /identity/v3/users/1b9e930927c345698d5ce806ff3173a4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:24.906709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:24.909284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:24.927673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c27c6-18f9-46cb-b154-8c606d293491 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:24.993171 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c353a5-4f67-4e10-951b-70596479080f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:24.994005 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c353a5-4f67-4e10-951b-70596479080f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:24.994335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c353a5-4f67-4e10-951b-70596479080f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:24.994658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c353a5-4f67-4e10-951b-70596479080f admin admin] PATH_INFO: `/v3/projects/ac2d218f7b3a47d6b96df5cd7871fb1b` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:24.996320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c353a5-4f67-4e10-951b-70596479080f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac2d218f7b3a47d6b96df5cd7871fb1b)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:24.999075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c353a5-4f67-4e10-951b-70596479080f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:25.055590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-88c353a5-4f67-4e10-951b-70596479080f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac2d218f7b3a47d6b96df5cd7871fb1b', 'request_id': 'req-88c353a5-4f67-4e10-951b-70596479080f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:25.062155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-88c353a5-4f67-4e10-951b-70596479080f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac2d218f7b3a47d6b96df5cd7871fb1b', 'request_id': 'req-88c353a5-4f67-4e10-951b-70596479080f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:25.067998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-88c353a5-4f67-4e10-951b-70596479080f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac2d218f7b3a47d6b96df5cd7871fb1b', 'request_id': 'req-88c353a5-4f67-4e10-951b-70596479080f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:25.083457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-88c353a5-4f67-4e10-951b-70596479080f admin admin] The token cache is being invalidate because project ac2d218f7b3a47d6b96df5cd7871fb1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:25.083743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-88c353a5-4f67-4e10-951b-70596479080f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:25.084374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-88c353a5-4f67-4e10-951b-70596479080f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:25.085876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1172/2350] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:24 2022] DELETE /identity/v3/projects/ac2d218f7b3a47d6b96df5cd7871fb1b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:25.091582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca03ea7-cbb4-4db7-ade6-5696f8ce99b4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:25.092098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca03ea7-cbb4-4db7-ade6-5696f8ce99b4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:25.092098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca03ea7-cbb4-4db7-ade6-5696f8ce99b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:25.111935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1ca03ea7-cbb4-4db7-ade6-5696f8ce99b4 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:25.113083 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1ca03ea7-cbb4-4db7-ade6-5696f8ce99b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:25.153620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:25.174255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1179/2351] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:25.182083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d431a-6c95-4b68-8a09-9b3d34454382 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:25.182441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d431a-6c95-4b68-8a09-9b3d34454382 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:25.182810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d431a-6c95-4b68-8a09-9b3d34454382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:25.203474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-792d431a-6c95-4b68-8a09-9b3d34454382 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:25.204690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-792d431a-6c95-4b68-8a09-9b3d34454382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:25.213985 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1173/2352] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:25.218740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca03ea7-cbb4-4db7-ade6-5696f8ce99b4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:25.229065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca03ea7-cbb4-4db7-ade6-5696f8ce99b4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:25.234395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41c60c-a779-4db1-bedd-6d8c51459a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:25.235005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41c60c-a779-4db1-bedd-6d8c51459a2c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:25.235282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41c60c-a779-4db1-bedd-6d8c51459a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:25.235550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41c60c-a779-4db1-bedd-6d8c51459a2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:25.236210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41c60c-a779-4db1-bedd-6d8c51459a2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:25.236510 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41c60c-a779-4db1-bedd-6d8c51459a2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:25.238170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41c60c-a779-4db1-bedd-6d8c51459a2c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:25.252311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1180/2353] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:25.257157 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d431a-6c95-4b68-8a09-9b3d34454382 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:25.266821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d431a-6c95-4b68-8a09-9b3d34454382 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:25.271721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2309e-928e-4bc1-9df3-0dc105e12537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:25.272318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2309e-928e-4bc1-9df3-0dc105e12537 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:25.272612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2309e-928e-4bc1-9df3-0dc105e12537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:25.272884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2309e-928e-4bc1-9df3-0dc105e12537 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:25.273523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2309e-928e-4bc1-9df3-0dc105e12537 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:25.275459 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2309e-928e-4bc1-9df3-0dc105e12537 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:25.299215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:25.309019 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1174/2354] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:25 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:25.310904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41c60c-a779-4db1-bedd-6d8c51459a2c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:25.321197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41c60c-a779-4db1-bedd-6d8c51459a2c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:25.379002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c005b-88df-46f1-a257-f5c2651727c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:25.379519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c005b-88df-46f1-a257-f5c2651727c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:25.379826 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c005b-88df-46f1-a257-f5c2651727c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:25.380085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c005b-88df-46f1-a257-f5c2651727c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:25.380754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c005b-88df-46f1-a257-f5c2651727c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:25.382767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c005b-88df-46f1-a257-f5c2651727c1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:25.406321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1181/2355] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:25 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:25.410640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2309e-928e-4bc1-9df3-0dc105e12537 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:25.421211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2309e-928e-4bc1-9df3-0dc105e12537 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:25.425947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8d12f-b0a4-4652-a5c0-4e508766737f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:25.426541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8d12f-b0a4-4652-a5c0-4e508766737f admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:25.426810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8d12f-b0a4-4652-a5c0-4e508766737f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:25.427072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8d12f-b0a4-4652-a5c0-4e508766737f admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:25.427741 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8d12f-b0a4-4652-a5c0-4e508766737f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:25.429358 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8d12f-b0a4-4652-a5c0-4e508766737f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:25.439401 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1175/2356] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:25.443494 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c005b-88df-46f1-a257-f5c2651727c1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:25.453961 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c005b-88df-46f1-a257-f5c2651727c1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:25.458959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc0fac-f94b-48ee-ace6-cb2a98312392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:25.459545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0fac-f94b-48ee-ace6-cb2a98312392 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:25.459818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0fac-f94b-48ee-ace6-cb2a98312392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:25.460054 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc0fac-f94b-48ee-ace6-cb2a98312392 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:25.460733 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc0fac-f94b-48ee-ace6-cb2a98312392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:25.462546 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc0fac-f94b-48ee-ace6-cb2a98312392 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:25.473901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1182/2357] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:25.478127 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8d12f-b0a4-4652-a5c0-4e508766737f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:25.488949 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8d12f-b0a4-4652-a5c0-4e508766737f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:25.495475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da89e1-7337-4143-a0bd-8cb5fb4343cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:25.496427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da89e1-7337-4143-a0bd-8cb5fb4343cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:25.496698 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da89e1-7337-4143-a0bd-8cb5fb4343cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:25.497001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da89e1-7337-4143-a0bd-8cb5fb4343cd admin admin] PATH_INFO: `/v3/projects/70c13ec7c4a243018586d0dc9d5cd8f1/users/ce80305a37e744028c579372856bbb48/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:25.512050 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da89e1-7337-4143-a0bd-8cb5fb4343cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c13ec7c4a243018586d0dc9d5cd8f1, user_id=ce80305a37e744028c579372856bbb48, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:25.514921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da89e1-7337-4143-a0bd-8cb5fb4343cd admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:25.530320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1176/2358] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:25 2022] PUT /identity/v3/projects/70c13ec7c4a243018586d0dc9d5cd8f1/users/ce80305a37e744028c579372856bbb48/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:25.535532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56040e74-9796-45d5-a34a-8c9a9d8a0786 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:25.535922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56040e74-9796-45d5-a34a-8c9a9d8a0786 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:25.536261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56040e74-9796-45d5-a34a-8c9a9d8a0786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:25.556092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-56040e74-9796-45d5-a34a-8c9a9d8a0786 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:25.557310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-56040e74-9796-45d5-a34a-8c9a9d8a0786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:25.617565 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1183/2359] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:25.627319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da89e1-7337-4143-a0bd-8cb5fb4343cd admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:25.629167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d9da89e1-7337-4143-a0bd-8cb5fb4343cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:25.693503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da89e1-7337-4143-a0bd-8cb5fb4343cd admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:25.699143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd76381-79b0-44ea-b78f-16ae311c30dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:25.699931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd76381-79b0-44ea-b78f-16ae311c30dc service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:25.700200 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd76381-79b0-44ea-b78f-16ae311c30dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:25.700463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd76381-79b0-44ea-b78f-16ae311c30dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:25.709529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd76381-79b0-44ea-b78f-16ae311c30dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:25.710950 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd76381-79b0-44ea-b78f-16ae311c30dc service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:25.720624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1177/2360] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:26.431406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d72b10-9223-4225-acd0-aa97168ee6c4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.431969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d72b10-9223-4225-acd0-aa97168ee6c4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.432163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d72b10-9223-4225-acd0-aa97168ee6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.464834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-05d72b10-9223-4225-acd0-aa97168ee6c4 None None] MFA Rules not processed for user `ce80305a37e744028c579372856bbb48`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:26.466304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-05d72b10-9223-4225-acd0-aa97168ee6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:26.512363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:26.559450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1184/2361] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:26.564047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd76381-79b0-44ea-b78f-16ae311c30dc service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:26.575995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd76381-79b0-44ea-b78f-16ae311c30dc service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:26.581146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ee8ad-16d7-49e3-94c9-af967080bdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:26.582000 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ee8ad-16d7-49e3-94c9-af967080bdbe admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.582465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ee8ad-16d7-49e3-94c9-af967080bdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.582936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ee8ad-16d7-49e3-94c9-af967080bdbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.583751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ee8ad-16d7-49e3-94c9-af967080bdbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:26.586075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ee8ad-16d7-49e3-94c9-af967080bdbe admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:26.624860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1178/2362] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:26 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:26.626862 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d72b10-9223-4225-acd0-aa97168ee6c4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:26.641424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d72b10-9223-4225-acd0-aa97168ee6c4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:26.716219 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f3594-8539-4f86-a001-e8839bc85582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:26.716882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f3594-8539-4f86-a001-e8839bc85582 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.717174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f3594-8539-4f86-a001-e8839bc85582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.717431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f3594-8539-4f86-a001-e8839bc85582 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.718088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f3594-8539-4f86-a001-e8839bc85582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:26.720050 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f3594-8539-4f86-a001-e8839bc85582 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:26.751898 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1185/2363] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:26 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:26.756238 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ee8ad-16d7-49e3-94c9-af967080bdbe admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:26.770874 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ee8ad-16d7-49e3-94c9-af967080bdbe admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:26.779070 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca5057-ea66-4e4d-bdc4-0842f4ea608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:26.779692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca5057-ea66-4e4d-bdc4-0842f4ea608e admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.780293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca5057-ea66-4e4d-bdc4-0842f4ea608e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.780293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca5057-ea66-4e4d-bdc4-0842f4ea608e admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.781258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca5057-ea66-4e4d-bdc4-0842f4ea608e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:26.783008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca5057-ea66-4e4d-bdc4-0842f4ea608e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:26.787713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f3594-8539-4f86-a001-e8839bc85582 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:26.791184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b20f3594-8539-4f86-a001-e8839bc85582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:26.807580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1179/2364] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:26.812422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca5057-ea66-4e4d-bdc4-0842f4ea608e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:26.825956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca5057-ea66-4e4d-bdc4-0842f4ea608e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:26.832561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67313b-4b76-48a0-8ebf-bd95525e9689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:26.833403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67313b-4b76-48a0-8ebf-bd95525e9689 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.833684 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67313b-4b76-48a0-8ebf-bd95525e9689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.833952 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67313b-4b76-48a0-8ebf-bd95525e9689 admin admin] PATH_INFO: `/v3/projects/55890ded45124e26a3a39a6d3b7647c7/users/1e773f9b9a5d4b67a616abe18e4b950f/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.854033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67313b-4b76-48a0-8ebf-bd95525e9689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55890ded45124e26a3a39a6d3b7647c7, user_id=1e773f9b9a5d4b67a616abe18e4b950f, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:26.857082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67313b-4b76-48a0-8ebf-bd95525e9689 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:26.874774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f3594-8539-4f86-a001-e8839bc85582 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:26.877115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1180/2365] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:26 2022] PUT /identity/v3/projects/55890ded45124e26a3a39a6d3b7647c7/users/1e773f9b9a5d4b67a616abe18e4b950f/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:26.878824 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87848d4f-3043-4bbd-9425-da9a7245a2bb None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.879310 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87848d4f-3043-4bbd-9425-da9a7245a2bb None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.880007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87848d4f-3043-4bbd-9425-da9a7245a2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.901497 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-87848d4f-3043-4bbd-9425-da9a7245a2bb None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:26.903357 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-87848d4f-3043-4bbd-9425-da9a7245a2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:26.929826 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d98ae-3682-4d06-815c-f188444be424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:26.930438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d98ae-3682-4d06-815c-f188444be424 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:26.930712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d98ae-3682-4d06-815c-f188444be424 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:26.931001 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d98ae-3682-4d06-815c-f188444be424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:26.932936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d82d98ae-3682-4d06-815c-f188444be424 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:26.967945 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1181/2366] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:21:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:26.969396 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87848d4f-3043-4bbd-9425-da9a7245a2bb None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:26.980820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:27.004073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d98ae-3682-4d06-815c-f188444be424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:27.005267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:27.006238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d98ae-3682-4d06-815c-f188444be424 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:27.018583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1186/2367] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:27.019359 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d98ae-3682-4d06-815c-f188444be424 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:27.021124 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d82d98ae-3682-4d06-815c-f188444be424 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:27.068745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87848d4f-3043-4bbd-9425-da9a7245a2bb None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:27.080844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9af04-61fa-4815-bcf1-c87e78b741c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:27.081481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9af04-61fa-4815-bcf1-c87e78b741c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:27.081754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9af04-61fa-4815-bcf1-c87e78b741c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:27.082022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9af04-61fa-4815-bcf1-c87e78b741c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:27.082680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9af04-61fa-4815-bcf1-c87e78b741c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:27.087338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9af04-61fa-4815-bcf1-c87e78b741c3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:27.115586 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1182/2368] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:27.123480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9af04-61fa-4815-bcf1-c87e78b741c3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:27.132068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d98ae-3682-4d06-815c-f188444be424 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:27.159739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effce60-ee17-4c90-ab0e-6a511040d3b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:27.159739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effce60-ee17-4c90-ab0e-6a511040d3b2 service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:27.159739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effce60-ee17-4c90-ab0e-6a511040d3b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:27.159739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effce60-ee17-4c90-ab0e-6a511040d3b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:27.163674 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9af04-61fa-4815-bcf1-c87e78b741c3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:27.168215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effce60-ee17-4c90-ab0e-6a511040d3b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:27.170132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effce60-ee17-4c90-ab0e-6a511040d3b2 service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:27.177095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f9b655-f0b8-4a22-a8fa-054ad911edab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:27.177655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9b655-f0b8-4a22-a8fa-054ad911edab admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:27.178097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9b655-f0b8-4a22-a8fa-054ad911edab admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:27.178477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f9b655-f0b8-4a22-a8fa-054ad911edab admin admin] PATH_INFO: `/v3/projects/55890ded45124e26a3a39a6d3b7647c7/users/1e773f9b9a5d4b67a616abe18e4b950f/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:27.180811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9b655-f0b8-4a22-a8fa-054ad911edab admin admin] RBAC: Authorizing `identity:create_grant(project_id=55890ded45124e26a3a39a6d3b7647c7, user_id=1e773f9b9a5d4b67a616abe18e4b950f, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:27.183608 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1187/2369] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:21:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:21:27.201808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f9b655-f0b8-4a22-a8fa-054ad911edab admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:27.237665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1183/2370] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:27 2022] PUT /identity/v3/projects/55890ded45124e26a3a39a6d3b7647c7/users/1e773f9b9a5d4b67a616abe18e4b950f/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:28.106213 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effce60-ee17-4c90-ab0e-6a511040d3b2 service swift] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:28.119385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effce60-ee17-4c90-ab0e-6a511040d3b2 service swift] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:28.191525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589393a-dde1-47a3-b04d-7db70f255fdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:28.192304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589393a-dde1-47a3-b04d-7db70f255fdb service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:28.192720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589393a-dde1-47a3-b04d-7db70f255fdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:28.193087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589393a-dde1-47a3-b04d-7db70f255fdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:28.195052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2589393a-dde1-47a3-b04d-7db70f255fdb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:28.276688 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed64891-f52f-4204-9c02-d09f1009172d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:28.277718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed64891-f52f-4204-9c02-d09f1009172d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:28.278004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed64891-f52f-4204-9c02-d09f1009172d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:28.326649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589393a-dde1-47a3-b04d-7db70f255fdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:28.328429 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589393a-dde1-47a3-b04d-7db70f255fdb service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:28.367365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1188/2371] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:28.383784 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2ed64891-f52f-4204-9c02-d09f1009172d None None] MFA Rules not processed for user `1e773f9b9a5d4b67a616abe18e4b950f`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:28.389899 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2ed64891-f52f-4204-9c02-d09f1009172d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:28.538101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1184/2372] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:28.552381 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589393a-dde1-47a3-b04d-7db70f255fdb service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:28.569329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589393a-dde1-47a3-b04d-7db70f255fdb service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:28.581992 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2035f2-7728-4771-a03e-f260300b1f3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:28.590297 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2035f2-7728-4771-a03e-f260300b1f3d service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:28.590597 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2035f2-7728-4771-a03e-f260300b1f3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:28.590988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2035f2-7728-4771-a03e-f260300b1f3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:28.624173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:28.676794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2035f2-7728-4771-a03e-f260300b1f3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:28.677472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2035f2-7728-4771-a03e-f260300b1f3d service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:28.702156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1189/2373] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:30.189668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed64891-f52f-4204-9c02-d09f1009172d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:30.191711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2ed64891-f52f-4204-9c02-d09f1009172d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:30.205240 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed64891-f52f-4204-9c02-d09f1009172d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:30.212404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad923a-3fc2-47da-b532-e8c967dc981e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:30.213253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad923a-3fc2-47da-b532-e8c967dc981e service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:30.216106 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad923a-3fc2-47da-b532-e8c967dc981e service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:30.216614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad923a-3fc2-47da-b532-e8c967dc981e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:30.227244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad923a-3fc2-47da-b532-e8c967dc981e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:30.228928 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad923a-3fc2-47da-b532-e8c967dc981e service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:30.243454 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1185/2374] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:38.683202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=19 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:38.791367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2035f2-7728-4771-a03e-f260300b1f3d service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:38.852691 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2035f2-7728-4771-a03e-f260300b1f3d service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:38.857845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931231d1-cca0-43a0-b76e-108be1757fd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:38.858776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931231d1-cca0-43a0-b76e-108be1757fd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:38.859038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931231d1-cca0-43a0-b76e-108be1757fd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:38.859299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931231d1-cca0-43a0-b76e-108be1757fd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:38.861117 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-931231d1-cca0-43a0-b76e-108be1757fd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:38.929929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931231d1-cca0-43a0-b76e-108be1757fd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:38.931306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931231d1-cca0-43a0-b76e-108be1757fd4 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:38.942135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1190/2375] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:39.710724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad923a-3fc2-47da-b532-e8c967dc981e service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:39.712196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7dad923a-3fc2-47da-b532-e8c967dc981e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:39.802242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad923a-3fc2-47da-b532-e8c967dc981e service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:39.808006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:39.809260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:39.809773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:39.810259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] PATH_INFO: `/v3/users/fb36dfeeac224133a00ae8c8282538bd` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:39.811902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb36dfeeac224133a00ae8c8282538bd)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:39.814132 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:39.831526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:39.888254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb36dfeeac224133a00ae8c8282538bd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:39.902806 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb36dfeeac224133a00ae8c8282538bd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:39.921962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb36dfeeac224133a00ae8c8282538bd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:39.934059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb36dfeeac224133a00ae8c8282538bd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:39.935079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb36dfeeac224133a00ae8c8282538bd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:39.953269 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1186/2376] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:39 2022] DELETE /identity/v3/users/fb36dfeeac224133a00ae8c8282538bd => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:39.954288 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957c913-dee7-4a9d-bdab-5166f784526f None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:39.954771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957c913-dee7-4a9d-bdab-5166f784526f None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:39.954862 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957c913-dee7-4a9d-bdab-5166f784526f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:40.004490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d957c913-dee7-4a9d-bdab-5166f784526f None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:40.005986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d957c913-dee7-4a9d-bdab-5166f784526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:40.086221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1191/2377] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:40.095312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:40.097192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:40.166908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680c79c-bbc0-4dae-9d79-148515b7d90d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:40.171168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb18dbf-3a21-4035-ae28-3c2ce7254982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:40.171864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb18dbf-3a21-4035-ae28-3c2ce7254982 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:40.172238 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb18dbf-3a21-4035-ae28-3c2ce7254982 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:40.172572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb18dbf-3a21-4035-ae28-3c2ce7254982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:40.186593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb18dbf-3a21-4035-ae28-3c2ce7254982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:40.188116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb18dbf-3a21-4035-ae28-3c2ce7254982 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:40.202748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1187/2378] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:21:40.538671 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957c913-dee7-4a9d-bdab-5166f784526f None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:40.544633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d957c913-dee7-4a9d-bdab-5166f784526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:40.556152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957c913-dee7-4a9d-bdab-5166f784526f None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:40.562294 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:40.563070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:40.563440 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:40.563799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] PATH_INFO: `/v3/projects/9ce3cc699fd54ce8af2a9864646b3481` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:40.565140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce3cc699fd54ce8af2a9864646b3481)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:40.567359 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:40.658676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:40.664055 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce3cc699fd54ce8af2a9864646b3481', 'request_id': 'req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:40.682260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce3cc699fd54ce8af2a9864646b3481', 'request_id': 'req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:40.697022 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce3cc699fd54ce8af2a9864646b3481', 'request_id': 'req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:40.726817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] The token cache is being invalidate because project 9ce3cc699fd54ce8af2a9864646b3481 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:40.727713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:40.728432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:40.732644 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1192/2379] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:40 2022] DELETE /identity/v3/projects/9ce3cc699fd54ce8af2a9864646b3481 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:40.735155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb18dbf-3a21-4035-ae28-3c2ce7254982 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:40.753089 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb18dbf-3a21-4035-ae28-3c2ce7254982 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:40.905802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:40.906901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:40.907436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:40.907927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] PATH_INFO: `/v3/users/3bf7ad9e23f44116bb24ef9fb77409f9` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:40.909791 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bf7ad9e23f44116bb24ef9fb77409f9)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:40.912263 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:40.995266 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:41.015438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bf7ad9e23f44116bb24ef9fb77409f9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:41.028052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bf7ad9e23f44116bb24ef9fb77409f9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:41.046434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bf7ad9e23f44116bb24ef9fb77409f9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:41.059665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bf7ad9e23f44116bb24ef9fb77409f9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:41.060815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bf7ad9e23f44116bb24ef9fb77409f9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:41.078365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1188/2380] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:40 2022] DELETE /identity/v3/users/3bf7ad9e23f44116bb24ef9fb77409f9 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:41.612391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:41.616186 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:41.646432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a20dc-8b60-4f7f-a58c-50e3a67ca42d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:41.798894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:41.799216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:41.799633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:41.800006 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] PATH_INFO: `/v3/projects/3565f08c24dc465fb9a57b84705cbe2d` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:41.801368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3565f08c24dc465fb9a57b84705cbe2d)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:41.803713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:41.878409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3565f08c24dc465fb9a57b84705cbe2d', 'request_id': 'req-01514d11-597e-4bb3-884a-f16a1d99fb04'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:41.889025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3565f08c24dc465fb9a57b84705cbe2d', 'request_id': 'req-01514d11-597e-4bb3-884a-f16a1d99fb04'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:41.898177 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3565f08c24dc465fb9a57b84705cbe2d', 'request_id': 'req-01514d11-597e-4bb3-884a-f16a1d99fb04'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:41.912169 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] The token cache is being invalidate because project 3565f08c24dc465fb9a57b84705cbe2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:41.912618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:41.913157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:41.914964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1193/2381] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:41 2022] DELETE /identity/v3/projects/3565f08c24dc465fb9a57b84705cbe2d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:41.918864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:41.920350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:41.935614 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f46ff-5e56-484d-bc95-9d2e52c1505e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:42.014368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c73bc1-06b9-4fe7-8dfc-f7749b3bd584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:42.015025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c73bc1-06b9-4fe7-8dfc-f7749b3bd584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:42.015304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c73bc1-06b9-4fe7-8dfc-f7749b3bd584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:42.015564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c73bc1-06b9-4fe7-8dfc-f7749b3bd584 admin admin] PATH_INFO: `/v3/users/211fce92d7e243b293c2f7e3641593cb` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:42.017018 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c73bc1-06b9-4fe7-8dfc-f7749b3bd584 admin admin] RBAC: Authorizing `identity:delete_user(user_id=211fce92d7e243b293c2f7e3641593cb)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:42.019565 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c73bc1-06b9-4fe7-8dfc-f7749b3bd584 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:42.121579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-17c73bc1-06b9-4fe7-8dfc-f7749b3bd584 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '211fce92d7e243b293c2f7e3641593cb'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:42.130222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-17c73bc1-06b9-4fe7-8dfc-f7749b3bd584 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '211fce92d7e243b293c2f7e3641593cb'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:42.151122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-17c73bc1-06b9-4fe7-8dfc-f7749b3bd584 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '211fce92d7e243b293c2f7e3641593cb'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:42.162766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-17c73bc1-06b9-4fe7-8dfc-f7749b3bd584 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '211fce92d7e243b293c2f7e3641593cb'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:42.164037 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-17c73bc1-06b9-4fe7-8dfc-f7749b3bd584 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '211fce92d7e243b293c2f7e3641593cb'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:42.173848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1189/2382] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:41 2022] DELETE /identity/v3/users/211fce92d7e243b293c2f7e3641593cb => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:42.796248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:42.798622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:42.828104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01514d11-597e-4bb3-884a-f16a1d99fb04 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:43.033069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f2233-6848-4998-8ab4-c00e80a10a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:43.033724 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f2233-6848-4998-8ab4-c00e80a10a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:43.033999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f2233-6848-4998-8ab4-c00e80a10a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:43.034265 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f2233-6848-4998-8ab4-c00e80a10a73 admin admin] PATH_INFO: `/v3/projects/f36e502ec40646c6a7741f7c49b54ed1` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:43.035482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f2233-6848-4998-8ab4-c00e80a10a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f36e502ec40646c6a7741f7c49b54ed1)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:43.037772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f2233-6848-4998-8ab4-c00e80a10a73 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:43.050819 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:43.173224 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-812f2233-6848-4998-8ab4-c00e80a10a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f36e502ec40646c6a7741f7c49b54ed1', 'request_id': 'req-812f2233-6848-4998-8ab4-c00e80a10a73'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:43.207389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-812f2233-6848-4998-8ab4-c00e80a10a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f36e502ec40646c6a7741f7c49b54ed1', 'request_id': 'req-812f2233-6848-4998-8ab4-c00e80a10a73'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:43.219947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-812f2233-6848-4998-8ab4-c00e80a10a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f36e502ec40646c6a7741f7c49b54ed1', 'request_id': 'req-812f2233-6848-4998-8ab4-c00e80a10a73'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:43.257928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-812f2233-6848-4998-8ab4-c00e80a10a73 admin admin] The token cache is being invalidate because project f36e502ec40646c6a7741f7c49b54ed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:43.258906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-812f2233-6848-4998-8ab4-c00e80a10a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:43.260795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-812f2233-6848-4998-8ab4-c00e80a10a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:43.265918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1194/2383] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:42 2022] DELETE /identity/v3/projects/f36e502ec40646c6a7741f7c49b54ed1 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:43.272720 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a78231-f4e4-4c90-90b4-ec33355f93a9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:43.273264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a78231-f4e4-4c90-90b4-ec33355f93a9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:43.273264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a78231-f4e4-4c90-90b4-ec33355f93a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:43.310625 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-98a78231-f4e4-4c90-90b4-ec33355f93a9 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:43.312337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-98a78231-f4e4-4c90-90b4-ec33355f93a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:43.367235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:43.466872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1190/2384] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:43.478009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e254d34-3cd7-404c-a17b-13be635a29a0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:43.478249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e254d34-3cd7-404c-a17b-13be635a29a0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:43.478433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e254d34-3cd7-404c-a17b-13be635a29a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:43.519521 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5e254d34-3cd7-404c-a17b-13be635a29a0 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:43.521873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5e254d34-3cd7-404c-a17b-13be635a29a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:43.555506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1195/2385] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:43.560752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a78231-f4e4-4c90-90b4-ec33355f93a9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:43.575242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a78231-f4e4-4c90-90b4-ec33355f93a9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:43.583855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f3389-b797-484b-b872-f70dfaf75b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:43.584541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f3389-b797-484b-b872-f70dfaf75b9e admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:43.584880 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f3389-b797-484b-b872-f70dfaf75b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:43.585489 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f3389-b797-484b-b872-f70dfaf75b9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:43.586075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f3389-b797-484b-b872-f70dfaf75b9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:43.586412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f3389-b797-484b-b872-f70dfaf75b9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:43.588263 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f3389-b797-484b-b872-f70dfaf75b9e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:43.633965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1191/2386] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:43.638909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e254d34-3cd7-404c-a17b-13be635a29a0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:43.651161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e254d34-3cd7-404c-a17b-13be635a29a0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:43.658431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554dedb-5806-4b68-b2c0-57c1a33b8356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:43.659593 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554dedb-5806-4b68-b2c0-57c1a33b8356 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:43.659871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554dedb-5806-4b68-b2c0-57c1a33b8356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:43.660136 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554dedb-5806-4b68-b2c0-57c1a33b8356 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:43.660823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554dedb-5806-4b68-b2c0-57c1a33b8356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:43.663004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5554dedb-5806-4b68-b2c0-57c1a33b8356 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:43.701904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1196/2387] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:43 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:43.706140 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f3389-b797-484b-b872-f70dfaf75b9e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:43.727255 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f3389-b797-484b-b872-f70dfaf75b9e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:43.805481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22f702-a6f8-4ed0-b462-7557737914e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:43.806354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22f702-a6f8-4ed0-b462-7557737914e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:43.806771 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22f702-a6f8-4ed0-b462-7557737914e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:43.807320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22f702-a6f8-4ed0-b462-7557737914e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:43.807982 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22f702-a6f8-4ed0-b462-7557737914e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:43.810059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22f702-a6f8-4ed0-b462-7557737914e8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:43.833657 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1192/2388] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:43 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:43.837703 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554dedb-5806-4b68-b2c0-57c1a33b8356 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:43.856311 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554dedb-5806-4b68-b2c0-57c1a33b8356 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:43.861787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f2ebf-1460-4240-9fbf-855c001c066c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:43.862444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f2ebf-1460-4240-9fbf-855c001c066c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:43.863135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f2ebf-1460-4240-9fbf-855c001c066c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:43.863566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f2ebf-1460-4240-9fbf-855c001c066c admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:43.864413 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f2ebf-1460-4240-9fbf-855c001c066c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:43.866483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f2ebf-1460-4240-9fbf-855c001c066c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:43.898224 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1197/2389] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:43.902008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22f702-a6f8-4ed0-b462-7557737914e8 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:43.919302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22f702-a6f8-4ed0-b462-7557737914e8 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:43.932331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d4738-b6b5-4ea9-b3a3-60e370a5e382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:43.933032 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d4738-b6b5-4ea9-b3a3-60e370a5e382 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:43.933565 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d4738-b6b5-4ea9-b3a3-60e370a5e382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:43.934708 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d4738-b6b5-4ea9-b3a3-60e370a5e382 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:43.935611 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d4738-b6b5-4ea9-b3a3-60e370a5e382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:43.937540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d4738-b6b5-4ea9-b3a3-60e370a5e382 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:43.957240 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1193/2390] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:43.962561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f2ebf-1460-4240-9fbf-855c001c066c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:43.976002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f2ebf-1460-4240-9fbf-855c001c066c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:43.986452 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bccfc-fe29-471f-b24f-f208ced224c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:43.987742 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bccfc-fe29-471f-b24f-f208ced224c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:43.988634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bccfc-fe29-471f-b24f-f208ced224c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:43.989056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76bccfc-fe29-471f-b24f-f208ced224c7 admin admin] PATH_INFO: `/v3/projects/87d09dadc90d4204998935316607f770/users/4341aa95ae774ca88223cbb8ff04d56a/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:44.010307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bccfc-fe29-471f-b24f-f208ced224c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d09dadc90d4204998935316607f770, user_id=4341aa95ae774ca88223cbb8ff04d56a, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:44.013789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76bccfc-fe29-471f-b24f-f208ced224c7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:44.031880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1198/2391] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:43 2022] PUT /identity/v3/projects/87d09dadc90d4204998935316607f770/users/4341aa95ae774ca88223cbb8ff04d56a/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:44.105518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08169e44-b3fd-4a18-ba60-880fce7ea839 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:44.105794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08169e44-b3fd-4a18-ba60-880fce7ea839 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:44.106083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08169e44-b3fd-4a18-ba60-880fce7ea839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:44.135004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-08169e44-b3fd-4a18-ba60-880fce7ea839 None None] MFA Rules not processed for user `4341aa95ae774ca88223cbb8ff04d56a`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:44.136235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-08169e44-b3fd-4a18-ba60-880fce7ea839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:44.261442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bccfc-fe29-471f-b24f-f208ced224c7 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:44.267076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1194/2392] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:44.338119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76bccfc-fe29-471f-b24f-f208ced224c7 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:44.343152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162aa8cf-c857-464a-aa4b-6020c889e36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:44.343756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aa8cf-c857-464a-aa4b-6020c889e36a admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:44.344033 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aa8cf-c857-464a-aa4b-6020c889e36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:44.344303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aa8cf-c857-464a-aa4b-6020c889e36a admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:44.344994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162aa8cf-c857-464a-aa4b-6020c889e36a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:44.346975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162aa8cf-c857-464a-aa4b-6020c889e36a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:44.375231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1199/2393] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:44 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:44.379497 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08169e44-b3fd-4a18-ba60-880fce7ea839 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:44.390861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08169e44-b3fd-4a18-ba60-880fce7ea839 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:44.415719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:44.449933 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963292c-628b-45cc-8a93-031ba00d6972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:44.450572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963292c-628b-45cc-8a93-031ba00d6972 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:44.451207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963292c-628b-45cc-8a93-031ba00d6972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:44.451648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963292c-628b-45cc-8a93-031ba00d6972 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:44.452352 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963292c-628b-45cc-8a93-031ba00d6972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:44.454838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963292c-628b-45cc-8a93-031ba00d6972 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:44.482761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1195/2394] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:44 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:44.484052 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162aa8cf-c857-464a-aa4b-6020c889e36a admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:44.494994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162aa8cf-c857-464a-aa4b-6020c889e36a admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:44.500209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4154a-d11d-4cb4-b8c6-d8cef27dc67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:44.500852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4154a-d11d-4cb4-b8c6-d8cef27dc67f admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:44.501127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4154a-d11d-4cb4-b8c6-d8cef27dc67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:44.501393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4154a-d11d-4cb4-b8c6-d8cef27dc67f admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:44.502043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4154a-d11d-4cb4-b8c6-d8cef27dc67f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:44.503670 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4154a-d11d-4cb4-b8c6-d8cef27dc67f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:44.514890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1200/2395] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:44.518879 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963292c-628b-45cc-8a93-031ba00d6972 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:44.529877 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963292c-628b-45cc-8a93-031ba00d6972 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:44.539245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb8aa0-26a6-4765-8615-0be37c93df90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:44.540418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb8aa0-26a6-4765-8615-0be37c93df90 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:44.541631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb8aa0-26a6-4765-8615-0be37c93df90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:44.541631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb8aa0-26a6-4765-8615-0be37c93df90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:44.542142 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb8aa0-26a6-4765-8615-0be37c93df90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:44.545230 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb8aa0-26a6-4765-8615-0be37c93df90 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:44.562471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1196/2396] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:44.563969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4154a-d11d-4cb4-b8c6-d8cef27dc67f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:44.570438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:44.574968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4154a-d11d-4cb4-b8c6-d8cef27dc67f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:44.580575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a60f4-3cd0-4bc6-a8f1-b6e6e0633ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:44.581611 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a60f4-3cd0-4bc6-a8f1-b6e6e0633ca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:44.582302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a60f4-3cd0-4bc6-a8f1-b6e6e0633ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:44.582667 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a60f4-3cd0-4bc6-a8f1-b6e6e0633ca4 admin admin] PATH_INFO: `/v3/projects/e2b1185535f0447c845592ecf969cbb3/users/fed054cb2e5d412c90360294e31fa41e/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:44.601634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a60f4-3cd0-4bc6-a8f1-b6e6e0633ca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b1185535f0447c845592ecf969cbb3, user_id=fed054cb2e5d412c90360294e31fa41e, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:44.604518 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a60f4-3cd0-4bc6-a8f1-b6e6e0633ca4 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:44.621990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1201/2397] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:44 2022] PUT /identity/v3/projects/e2b1185535f0447c845592ecf969cbb3/users/fed054cb2e5d412c90360294e31fa41e/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:44.632802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341612e-80df-4dac-a307-9506b2b1a90b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:44.632802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341612e-80df-4dac-a307-9506b2b1a90b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:44.633082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341612e-80df-4dac-a307-9506b2b1a90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:44.670405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6341612e-80df-4dac-a307-9506b2b1a90b None None] MFA Rules not processed for user `fed054cb2e5d412c90360294e31fa41e`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:44.671897 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6341612e-80df-4dac-a307-9506b2b1a90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:44.752122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1197/2398] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:44.753663 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a60f4-3cd0-4bc6-a8f1-b6e6e0633ca4 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:44.847140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a60f4-3cd0-4bc6-a8f1-b6e6e0633ca4 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:44.852358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1501f-e1d1-45a7-a712-c7f8b854128c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:44.853262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1501f-e1d1-45a7-a712-c7f8b854128c admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:44.853748 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1501f-e1d1-45a7-a712-c7f8b854128c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:44.853941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1501f-e1d1-45a7-a712-c7f8b854128c admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:44.854663 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1501f-e1d1-45a7-a712-c7f8b854128c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:44.857146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1501f-e1d1-45a7-a712-c7f8b854128c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:44.907493 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1202/2399] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:44 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:44.911931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341612e-80df-4dac-a307-9506b2b1a90b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:44.928280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341612e-80df-4dac-a307-9506b2b1a90b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:45.035295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc651704-2d73-47b3-ae53-367ec6a7c71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:45.036055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc651704-2d73-47b3-ae53-367ec6a7c71f admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:45.036403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc651704-2d73-47b3-ae53-367ec6a7c71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:45.036714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc651704-2d73-47b3-ae53-367ec6a7c71f admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:45.037366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc651704-2d73-47b3-ae53-367ec6a7c71f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:45.039712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc651704-2d73-47b3-ae53-367ec6a7c71f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:45.067930 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1198/2400] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:44 2022] POST /identity/v3/users => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:45.071203 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1501f-e1d1-45a7-a712-c7f8b854128c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:45.083825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1501f-e1d1-45a7-a712-c7f8b854128c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:45.090728 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ace7b-0c6e-4070-90f2-80cc5d40173c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:45.091361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ace7b-0c6e-4070-90f2-80cc5d40173c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:45.091640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ace7b-0c6e-4070-90f2-80cc5d40173c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:45.091910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ace7b-0c6e-4070-90f2-80cc5d40173c admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:45.092726 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ace7b-0c6e-4070-90f2-80cc5d40173c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:45.094629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ace7b-0c6e-4070-90f2-80cc5d40173c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:45.109895 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1203/2401] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:45.111572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc651704-2d73-47b3-ae53-367ec6a7c71f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:45.128225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc651704-2d73-47b3-ae53-367ec6a7c71f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:45.134054 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c269758-1858-4ace-aae2-45e733b55817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:45.134895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c269758-1858-4ace-aae2-45e733b55817 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:45.135188 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c269758-1858-4ace-aae2-45e733b55817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:45.135450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c269758-1858-4ace-aae2-45e733b55817 admin admin] PATH_INFO: `/v3/projects/9ff457e70fb94b64bc643104d79ef8f7/users/07ed99eb59b9478d98406d8e68aebbbd/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:45.158328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c269758-1858-4ace-aae2-45e733b55817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff457e70fb94b64bc643104d79ef8f7, user_id=07ed99eb59b9478d98406d8e68aebbbd, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:45.161343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c269758-1858-4ace-aae2-45e733b55817 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:45.177111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:45.188550 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1199/2402] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:45 2022] PUT /identity/v3/projects/9ff457e70fb94b64bc643104d79ef8f7/users/07ed99eb59b9478d98406d8e68aebbbd/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:45.193042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ace7b-0c6e-4070-90f2-80cc5d40173c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:45.205770 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ace7b-0c6e-4070-90f2-80cc5d40173c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:45.285352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d51310-611d-4a2f-bd70-800002c41ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:45.285871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d51310-611d-4a2f-bd70-800002c41ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:45.286141 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d51310-611d-4a2f-bd70-800002c41ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:45.286448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d51310-611d-4a2f-bd70-800002c41ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:45.287100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d51310-611d-4a2f-bd70-800002c41ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:45.288827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d51310-611d-4a2f-bd70-800002c41ccc admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:45.302687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1204/2403] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:45.307252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c269758-1858-4ace-aae2-45e733b55817 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:45.319121 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c269758-1858-4ace-aae2-45e733b55817 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:45.326487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f37f9-abe7-463e-9236-06bd679e6c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:45.327808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f37f9-abe7-463e-9236-06bd679e6c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:45.328086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f37f9-abe7-463e-9236-06bd679e6c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:45.328363 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f37f9-abe7-463e-9236-06bd679e6c4a admin admin] PATH_INFO: `/v3/projects/9ff457e70fb94b64bc643104d79ef8f7/users/07ed99eb59b9478d98406d8e68aebbbd/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:45.330808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f37f9-abe7-463e-9236-06bd679e6c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff457e70fb94b64bc643104d79ef8f7, user_id=07ed99eb59b9478d98406d8e68aebbbd, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:45.333761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f37f9-abe7-463e-9236-06bd679e6c4a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:45.348632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1200/2404] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:45 2022] PUT /identity/v3/projects/9ff457e70fb94b64bc643104d79ef8f7/users/07ed99eb59b9478d98406d8e68aebbbd/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:45.359907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a9bc5-87b5-4879-b7e7-91e7c8f9ce9b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:45.360260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a9bc5-87b5-4879-b7e7-91e7c8f9ce9b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:45.360896 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a9bc5-87b5-4879-b7e7-91e7c8f9ce9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:45.402173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4c9a9bc5-87b5-4879-b7e7-91e7c8f9ce9b None None] MFA Rules not processed for user `07ed99eb59b9478d98406d8e68aebbbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:45.403779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4c9a9bc5-87b5-4879-b7e7-91e7c8f9ce9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:45.467262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:45.514998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1205/2405] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:45.525172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f37f9-abe7-463e-9236-06bd679e6c4a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:45.527706 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-530f37f9-abe7-463e-9236-06bd679e6c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:45.646460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f37f9-abe7-463e-9236-06bd679e6c4a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:45.654909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfb682-05d4-48c9-808f-f426f14de0dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:45.657144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfb682-05d4-48c9-808f-f426f14de0dc service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:45.659201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfb682-05d4-48c9-808f-f426f14de0dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:45.660183 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bfb682-05d4-48c9-808f-f426f14de0dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:45.787181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bfb682-05d4-48c9-808f-f426f14de0dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:45.792097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bfb682-05d4-48c9-808f-f426f14de0dc service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:45.811518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1201/2406] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:46.279564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efc79f-f463-4b1d-a51d-66077e95e161 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:46.280436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efc79f-f463-4b1d-a51d-66077e95e161 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:46.280763 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efc79f-f463-4b1d-a51d-66077e95e161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:46.302899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-56efc79f-f463-4b1d-a51d-66077e95e161 None None] MFA Rules not processed for user `4cfac62e1c304167abb971815d5da358`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:46.304030 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-56efc79f-f463-4b1d-a51d-66077e95e161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:46.360732 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1206/2407] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:48.589957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfb682-05d4-48c9-808f-f426f14de0dc service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:48.602864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bfb682-05d4-48c9-808f-f426f14de0dc service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:48.616729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d369f-3ac1-4a0e-b9b5-6b20ea7f56bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:48.616729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d369f-3ac1-4a0e-b9b5-6b20ea7f56bb service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:48.616729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d369f-3ac1-4a0e-b9b5-6b20ea7f56bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:48.616729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d369f-3ac1-4a0e-b9b5-6b20ea7f56bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:48.710223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d369f-3ac1-4a0e-b9b5-6b20ea7f56bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:48.711901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d369f-3ac1-4a0e-b9b5-6b20ea7f56bb service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:48.726873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1202/2408] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:49.416779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56efc79f-f463-4b1d-a51d-66077e95e161 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:49.428694 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56efc79f-f463-4b1d-a51d-66077e95e161 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:49.432510 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf432890-3c9e-4847-83ed-34b85a03f360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:49.433146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432890-3c9e-4847-83ed-34b85a03f360 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:49.433649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432890-3c9e-4847-83ed-34b85a03f360 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:49.433914 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432890-3c9e-4847-83ed-34b85a03f360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:49.444214 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf432890-3c9e-4847-83ed-34b85a03f360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:49.446002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf432890-3c9e-4847-83ed-34b85a03f360 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:49.455441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1207/2409] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:51.590438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33170444-2af6-46ba-9194-b5ccf641bd8a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:51.590970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33170444-2af6-46ba-9194-b5ccf641bd8a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:51.590970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33170444-2af6-46ba-9194-b5ccf641bd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:51.612846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-33170444-2af6-46ba-9194-b5ccf641bd8a None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:51.614309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-33170444-2af6-46ba-9194-b5ccf641bd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:51.665195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:51.690013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1203/2410] 10.222.0.75 () {58 vars in 967 bytes} [Fri Sep 23 15:21:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:51.703563 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf432890-3c9e-4847-83ed-34b85a03f360 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:51.704983 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-cf432890-3c9e-4847-83ed-34b85a03f360 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:51.774145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf432890-3c9e-4847-83ed-34b85a03f360 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:51.780085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34bf2f5-a6d9-4aa9-8188-09ed27dc98f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:51.781302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bf2f5-a6d9-4aa9-8188-09ed27dc98f9 service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:51.781580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bf2f5-a6d9-4aa9-8188-09ed27dc98f9 service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:51.781844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bf2f5-a6d9-4aa9-8188-09ed27dc98f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:51.791178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34bf2f5-a6d9-4aa9-8188-09ed27dc98f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:51.793032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34bf2f5-a6d9-4aa9-8188-09ed27dc98f9 service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:51.802264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1208/2411] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:21:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 15:21:53.200284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d7e82-d7b1-4e15-a73c-834373e8bb52 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:53.200838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d7e82-d7b1-4e15-a73c-834373e8bb52 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:53.200838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d7e82-d7b1-4e15-a73c-834373e8bb52 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:53.202539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1204/2412] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:21:53.245195 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0e495-685b-4fc9-a125-5a13157fa477 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:53.245820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0e495-685b-4fc9-a125-5a13157fa477 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:53.245820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0e495-685b-4fc9-a125-5a13157fa477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:53.273216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-1fd0e495-685b-4fc9-a125-5a13157fa477 None None] MFA Rules not processed for user `2b8be1ec4a7a45b88d84bd9c48fab407`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:53.275152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1fd0e495-685b-4fc9-a125-5a13157fa477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:53.353982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:53.376820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1209/2413] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:21:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:53.403441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d7e82-d7b1-4e15-a73c-834373e8bb52 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:53.406767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0a2d7e82-d7b1-4e15-a73c-834373e8bb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:53.435977 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d7e82-d7b1-4e15-a73c-834373e8bb52 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:53.441502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d26d51-8fdb-4239-8581-31315bf59626 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:53.442294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d26d51-8fdb-4239-8581-31315bf59626 service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:53.442637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d26d51-8fdb-4239-8581-31315bf59626 service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:53.442968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d26d51-8fdb-4239-8581-31315bf59626 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:53.465768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d26d51-8fdb-4239-8581-31315bf59626 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:53.467449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d26d51-8fdb-4239-8581-31315bf59626 service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:53.512130 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1205/2414] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:53.528740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0e495-685b-4fc9-a125-5a13157fa477 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:53.636241 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0e495-685b-4fc9-a125-5a13157fa477 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:53.642896 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:53.643681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:53.644219 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:53.644710 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] PATH_INFO: `/v3/users/4341aa95ae774ca88223cbb8ff04d56a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:53.646072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4341aa95ae774ca88223cbb8ff04d56a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:53.648171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:53.708282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4341aa95ae774ca88223cbb8ff04d56a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:53.709526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4341aa95ae774ca88223cbb8ff04d56a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:53.733689 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4341aa95ae774ca88223cbb8ff04d56a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:53.757872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4341aa95ae774ca88223cbb8ff04d56a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:53.758619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4341aa95ae774ca88223cbb8ff04d56a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:53.774424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1210/2415] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:53 2022] DELETE /identity/v3/users/4341aa95ae774ca88223cbb8ff04d56a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:53.779932 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf56994-79e3-435f-afe6-41b4b54860eb None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:53.781135 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf56994-79e3-435f-afe6-41b4b54860eb None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:53.781650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf56994-79e3-435f-afe6-41b4b54860eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:53.814772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-dbf56994-79e3-435f-afe6-41b4b54860eb None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:53.818495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dbf56994-79e3-435f-afe6-41b4b54860eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:53.894604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1206/2416] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:53.918800 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:53.921893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:54.032740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301bbcc0-d1bf-4da4-bdcf-dbdab94d3f9c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:54.037974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c9202-e9ca-42aa-af69-3a77978c9173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:54.041153 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c9202-e9ca-42aa-af69-3a77978c9173 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:54.041599 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c9202-e9ca-42aa-af69-3a77978c9173 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:54.041969 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92c9202-e9ca-42aa-af69-3a77978c9173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:54.055859 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c9202-e9ca-42aa-af69-3a77978c9173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:54.057566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92c9202-e9ca-42aa-af69-3a77978c9173 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:54.070539 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1211/2417] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:54.467548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf56994-79e3-435f-afe6-41b4b54860eb None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:54.469804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dbf56994-79e3-435f-afe6-41b4b54860eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:54.482461 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf56994-79e3-435f-afe6-41b4b54860eb None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:54.488289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:54.489253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:54.489532 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:54.489801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] PATH_INFO: `/v3/users/ce80305a37e744028c579372856bbb48` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:54.492058 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce80305a37e744028c579372856bbb48)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:54.494451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:54.509172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c9202-e9ca-42aa-af69-3a77978c9173 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:54.513092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e92c9202-e9ca-42aa-af69-3a77978c9173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:54.533125 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92c9202-e9ca-42aa-af69-3a77978c9173 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:54.538536 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:54.539213 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:54.539450 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:54.539719 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] PATH_INFO: `/v3/users/159528b6ad49481f86bfbbac6187d584` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:54.541205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=159528b6ad49481f86bfbbac6187d584)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:54.547946 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:54.567615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:54.617794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce80305a37e744028c579372856bbb48'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:54.632008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce80305a37e744028c579372856bbb48'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:54.647347 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce80305a37e744028c579372856bbb48'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:54.657501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce80305a37e744028c579372856bbb48'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:54.660533 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce80305a37e744028c579372856bbb48'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:54.661121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '159528b6ad49481f86bfbbac6187d584'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:54.661723 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '159528b6ad49481f86bfbbac6187d584'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:54.669907 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '159528b6ad49481f86bfbbac6187d584'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:54.672282 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1207/2418] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:54 2022] DELETE /identity/v3/users/ce80305a37e744028c579372856bbb48 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:54.696085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '159528b6ad49481f86bfbbac6187d584'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:54.697666 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '159528b6ad49481f86bfbbac6187d584'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:54.738700 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1212/2419] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:54 2022] DELETE /identity/v3/users/159528b6ad49481f86bfbbac6187d584 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:54.873840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:54.875494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:54.890703 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:54.893528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:54.986260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:55.017553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea229c39-5e2b-4943-8ca9-59c1116d0068 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:55.021704 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad753e71-4f05-4730-9714-317023635b59 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:55.022506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:55.023475 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.023760 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.024169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] PATH_INFO: `/v3/projects/87d09dadc90d4204998935316607f770` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.025421 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d09dadc90d4204998935316607f770)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:55.027599 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:55.028932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:55.029719 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.030163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.036560 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] PATH_INFO: `/v3/projects/70c13ec7c4a243018586d0dc9d5cd8f1` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.037931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=70c13ec7c4a243018586d0dc9d5cd8f1)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:55.043810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:55.136992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d09dadc90d4204998935316607f770', 'request_id': 'req-62a47b07-6c4f-4028-a215-63b90d499a12'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.137492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70c13ec7c4a243018586d0dc9d5cd8f1', 'request_id': 'req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.147582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70c13ec7c4a243018586d0dc9d5cd8f1', 'request_id': 'req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.150815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d09dadc90d4204998935316607f770', 'request_id': 'req-62a47b07-6c4f-4028-a215-63b90d499a12'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.155244 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70c13ec7c4a243018586d0dc9d5cd8f1', 'request_id': 'req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.165300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d09dadc90d4204998935316607f770', 'request_id': 'req-62a47b07-6c4f-4028-a215-63b90d499a12'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.168122 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] The token cache is being invalidate because project 70c13ec7c4a243018586d0dc9d5cd8f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:55.168464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.169268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.170827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1213/2420] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:54 2022] DELETE /identity/v3/projects/70c13ec7c4a243018586d0dc9d5cd8f1 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:55.175126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:55.176857 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:55.182095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] The token cache is being invalidate because project 87d09dadc90d4204998935316607f770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:55.182095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.182349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.184000 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1208/2421] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:54 2022] DELETE /identity/v3/projects/87d09dadc90d4204998935316607f770 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:55.184833 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:55.186143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:55.200885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:55.259799 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd90b6-4c7a-4f83-94ec-5ee1971d75cd admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:55.267528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:55.268538 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.268877 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.269232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] PATH_INFO: `/v3/projects/efa8ccdc30dc4651b730950a0a5fe8d4` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.270524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=efa8ccdc30dc4651b730950a0a5fe8d4)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:55.270945 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47b07-6c4f-4028-a215-63b90d499a12 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:55.272793 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:55.276541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496d812-bc11-44c3-9e53-dc1ffb6f5216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:55.277439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496d812-bc11-44c3-9e53-dc1ffb6f5216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.278223 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496d812-bc11-44c3-9e53-dc1ffb6f5216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.278567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496d812-bc11-44c3-9e53-dc1ffb6f5216 admin admin] PATH_INFO: `/v3/users/1e773f9b9a5d4b67a616abe18e4b950f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.279825 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496d812-bc11-44c3-9e53-dc1ffb6f5216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e773f9b9a5d4b67a616abe18e4b950f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:55.282029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496d812-bc11-44c3-9e53-dc1ffb6f5216 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:55.335947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efa8ccdc30dc4651b730950a0a5fe8d4', 'request_id': 'req-3479bf8c-8761-4b42-95d5-08d312ee84ed'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.341313 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6496d812-bc11-44c3-9e53-dc1ffb6f5216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e773f9b9a5d4b67a616abe18e4b950f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.346345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efa8ccdc30dc4651b730950a0a5fe8d4', 'request_id': 'req-3479bf8c-8761-4b42-95d5-08d312ee84ed'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.349629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6496d812-bc11-44c3-9e53-dc1ffb6f5216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e773f9b9a5d4b67a616abe18e4b950f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.353296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efa8ccdc30dc4651b730950a0a5fe8d4', 'request_id': 'req-3479bf8c-8761-4b42-95d5-08d312ee84ed'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.362833 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6496d812-bc11-44c3-9e53-dc1ffb6f5216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e773f9b9a5d4b67a616abe18e4b950f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.365356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] The token cache is being invalidate because project efa8ccdc30dc4651b730950a0a5fe8d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:55.365981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.366657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.368543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1214/2422] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:55 2022] DELETE /identity/v3/projects/efa8ccdc30dc4651b730950a0a5fe8d4 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:55.369931 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:55.373841 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:55.375503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:55.378205 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6496d812-bc11-44c3-9e53-dc1ffb6f5216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e773f9b9a5d4b67a616abe18e4b950f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.379138 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-6496d812-bc11-44c3-9e53-dc1ffb6f5216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e773f9b9a5d4b67a616abe18e4b950f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.389870 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1209/2423] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:55 2022] DELETE /identity/v3/users/1e773f9b9a5d4b67a616abe18e4b950f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:55.392938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd84003-a5be-46ef-9185-0dfd6e7ef080 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.392938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd84003-a5be-46ef-9185-0dfd6e7ef080 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.392938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd84003-a5be-46ef-9185-0dfd6e7ef080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.416293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4bd84003-a5be-46ef-9185-0dfd6e7ef080 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:55.417343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4bd84003-a5be-46ef-9185-0dfd6e7ef080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:55.475484 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479bf8c-8761-4b42-95d5-08d312ee84ed admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:55.506859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1210/2424] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:55.509212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f40623-f839-42f5-a165-329ed2ff1ad7 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.509585 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f40623-f839-42f5-a165-329ed2ff1ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.509918 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f40623-f839-42f5-a165-329ed2ff1ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.531117 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-30f40623-f839-42f5-a165-329ed2ff1ad7 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:55.532494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-30f40623-f839-42f5-a165-329ed2ff1ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:55.540464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1211/2425] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:55.544868 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f40623-f839-42f5-a165-329ed2ff1ad7 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:55.547506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:55.556900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f40623-f839-42f5-a165-329ed2ff1ad7 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:55.559188 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:55.560326 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.561105 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.561484 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] PATH_INFO: `/v3/users/fed054cb2e5d412c90360294e31fa41e` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.561861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc275452-18e9-4191-a996-1a77f448818f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:55.562466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc275452-18e9-4191-a996-1a77f448818f admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.562973 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc275452-18e9-4191-a996-1a77f448818f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.563233 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc275452-18e9-4191-a996-1a77f448818f admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.563432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fed054cb2e5d412c90360294e31fa41e)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:55.563882 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc275452-18e9-4191-a996-1a77f448818f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:55.564179 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc275452-18e9-4191-a996-1a77f448818f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:55.565792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc275452-18e9-4191-a996-1a77f448818f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:55.566532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:55.582795 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1212/2426] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:55.584260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc275452-18e9-4191-a996-1a77f448818f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:55.594279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc275452-18e9-4191-a996-1a77f448818f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:55.599376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac46787-fc66-4f36-b6f6-aec46ebfc0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:55.600327 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46787-fc66-4f36-b6f6-aec46ebfc0be admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.600821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46787-fc66-4f36-b6f6-aec46ebfc0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.601122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46787-fc66-4f36-b6f6-aec46ebfc0be admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.601812 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac46787-fc66-4f36-b6f6-aec46ebfc0be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:55.603741 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac46787-fc66-4f36-b6f6-aec46ebfc0be admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:55.620424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fed054cb2e5d412c90360294e31fa41e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.621582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fed054cb2e5d412c90360294e31fa41e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.641589 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fed054cb2e5d412c90360294e31fa41e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.664932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fed054cb2e5d412c90360294e31fa41e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.665692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fed054cb2e5d412c90360294e31fa41e'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.670141 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1213/2427] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:55 2022] POST /identity/v3/projects => generated 389 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:21:55.673418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac46787-fc66-4f36-b6f6-aec46ebfc0be admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:55.676235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1ac46787-fc66-4f36-b6f6-aec46ebfc0be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:55.686896 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1215/2428] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:55 2022] DELETE /identity/v3/users/fed054cb2e5d412c90360294e31fa41e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:55.737988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:55.739471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:55.767692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac46787-fc66-4f36-b6f6-aec46ebfc0be admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:55.838723 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3277e-d469-415f-9d00-8c2295fb8ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:55.839433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3277e-d469-415f-9d00-8c2295fb8ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.839737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b6c1a-c7d4-45d9-bceb-c7da25ed077e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:55.839860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3277e-d469-415f-9d00-8c2295fb8ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.840155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3277e-d469-415f-9d00-8c2295fb8ed4 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.840902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3277e-d469-415f-9d00-8c2295fb8ed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:55.842855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3277e-d469-415f-9d00-8c2295fb8ed4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:55.844922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:55.845619 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.846549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.846549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd admin admin] PATH_INFO: `/v3/projects/55890ded45124e26a3a39a6d3b7647c7` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.847408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd admin admin] RBAC: Authorizing `identity:delete_project(project_id=55890ded45124e26a3a39a6d3b7647c7)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:55.849493 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:55.858793 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:55.885074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1214/2429] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:55 2022] POST /identity/v3/users => generated 295 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:55.886945 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3277e-d469-415f-9d00-8c2295fb8ed4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:55.900019 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3277e-d469-415f-9d00-8c2295fb8ed4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:55.936162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55890ded45124e26a3a39a6d3b7647c7', 'request_id': 'req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.949224 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55890ded45124e26a3a39a6d3b7647c7', 'request_id': 'req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.960053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55890ded45124e26a3a39a6d3b7647c7', 'request_id': 'req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.978274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd admin admin] The token cache is being invalidate because project 55890ded45124e26a3a39a6d3b7647c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:55.978841 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.979429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e08337-8e62-4b2b-aa32-4a20400921c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:55.979852 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-ab4e90c3-133e-4542-9c2a-8ac1c8a23acd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:55.980287 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e08337-8e62-4b2b-aa32-4a20400921c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.980677 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e08337-8e62-4b2b-aa32-4a20400921c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.981036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e08337-8e62-4b2b-aa32-4a20400921c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.981781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e08337-8e62-4b2b-aa32-4a20400921c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:55.982434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1216/2430] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:55 2022] DELETE /identity/v3/projects/55890ded45124e26a3a39a6d3b7647c7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:55.983958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e08337-8e62-4b2b-aa32-4a20400921c2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:55.987964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512ad72-489b-4893-8b5d-3682ddfc4219 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:55.988428 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512ad72-489b-4893-8b5d-3682ddfc4219 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:55.988886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512ad72-489b-4893-8b5d-3682ddfc4219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:55.995973 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1215/2431] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:55.999864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e08337-8e62-4b2b-aa32-4a20400921c2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.002501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:56.002825 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-05e08337-8e62-4b2b-aa32-4a20400921c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:56.017401 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e08337-8e62-4b2b-aa32-4a20400921c2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:56.019425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7512ad72-489b-4893-8b5d-3682ddfc4219 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:56.021505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7512ad72-489b-4893-8b5d-3682ddfc4219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:56.022595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a35724-46c0-46c1-93e1-8fd6b2e83c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:56.023204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a35724-46c0-46c1-93e1-8fd6b2e83c73 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.023517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a35724-46c0-46c1-93e1-8fd6b2e83c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.023742 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a35724-46c0-46c1-93e1-8fd6b2e83c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.024391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a35724-46c0-46c1-93e1-8fd6b2e83c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:56.026325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a35724-46c0-46c1-93e1-8fd6b2e83c73 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:56.032064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1217/2432] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:56.041476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0841294-f9e8-4d7e-b2c0-8be51a717d19 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.041826 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0841294-f9e8-4d7e-b2c0-8be51a717d19 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.042171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0841294-f9e8-4d7e-b2c0-8be51a717d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.047482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1216/2433] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:56.048926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a35724-46c0-46c1-93e1-8fd6b2e83c73 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.064502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a35724-46c0-46c1-93e1-8fd6b2e83c73 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:56.070331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae290ce-cf5d-49a3-bfb8-6019df86f344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:56.070998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae290ce-cf5d-49a3-bfb8-6019df86f344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.071271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae290ce-cf5d-49a3-bfb8-6019df86f344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.071813 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae290ce-cf5d-49a3-bfb8-6019df86f344 admin admin] PATH_INFO: `/v3/projects/9a5275503f8746648f26bd0d9bef8610/users/a2567a5095b4423eb37803483977f8d4/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.075414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a0841294-f9e8-4d7e-b2c0-8be51a717d19 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:56.076817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a0841294-f9e8-4d7e-b2c0-8be51a717d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:56.086062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1218/2434] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:21:56.087816 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae290ce-cf5d-49a3-bfb8-6019df86f344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a5275503f8746648f26bd0d9bef8610, user_id=a2567a5095b4423eb37803483977f8d4, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:56.090168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0841294-f9e8-4d7e-b2c0-8be51a717d19 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.090528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae290ce-cf5d-49a3-bfb8-6019df86f344 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:56.101479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0841294-f9e8-4d7e-b2c0-8be51a717d19 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:56.105970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1217/2435] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:56 2022] PUT /identity/v3/projects/9a5275503f8746648f26bd0d9bef8610/users/a2567a5095b4423eb37803483977f8d4/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:21:56.184968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe06d60-149d-4d3f-ac77-0c26aa6c3a47 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.184968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe06d60-149d-4d3f-ac77-0c26aa6c3a47 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.184968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe06d60-149d-4d3f-ac77-0c26aa6c3a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.213181 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78c886-06fb-4dfb-b3a9-18217ead3443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:56.213745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8fe06d60-149d-4d3f-ac77-0c26aa6c3a47 None None] MFA Rules not processed for user `a2567a5095b4423eb37803483977f8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:56.213906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78c886-06fb-4dfb-b3a9-18217ead3443 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.214346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78c886-06fb-4dfb-b3a9-18217ead3443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.214856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb78c886-06fb-4dfb-b3a9-18217ead3443 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.215629 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb78c886-06fb-4dfb-b3a9-18217ead3443 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:21:56.215998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb78c886-06fb-4dfb-b3a9-18217ead3443 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:56.218848 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8fe06d60-149d-4d3f-ac77-0c26aa6c3a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:56.222668 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb78c886-06fb-4dfb-b3a9-18217ead3443 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:56.241855 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1219/2436] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:21:56.242747 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78c886-06fb-4dfb-b3a9-18217ead3443 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.244419 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fb78c886-06fb-4dfb-b3a9-18217ead3443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:56.257800 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb78c886-06fb-4dfb-b3a9-18217ead3443 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:56.263330 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:56.263955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.264228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.264493 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] PATH_INFO: `/v3/projects/e2b1185535f0447c845592ecf969cbb3` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.265827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b1185535f0447c845592ecf969cbb3)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:56.267725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:56.290758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1218/2437] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:56.291731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe06d60-149d-4d3f-ac77-0c26aa6c3a47 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.305375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe06d60-149d-4d3f-ac77-0c26aa6c3a47 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:56.318880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b1185535f0447c845592ecf969cbb3', 'request_id': 'req-10740af5-a0b2-468b-b661-e300486ab9dd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.328195 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b1185535f0447c845592ecf969cbb3', 'request_id': 'req-10740af5-a0b2-468b-b661-e300486ab9dd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.336857 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b1185535f0447c845592ecf969cbb3', 'request_id': 'req-10740af5-a0b2-468b-b661-e300486ab9dd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.346026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:56.348275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] The token cache is being invalidate because project e2b1185535f0447c845592ecf969cbb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:56.348656 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.349427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.351009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1220/2438] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:56 2022] DELETE /identity/v3/projects/e2b1185535f0447c845592ecf969cbb3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:56.351770 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.353071 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:56.366811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaa4df-f39e-4858-a5cd-fd08e1ccf1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:56.367451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaa4df-f39e-4858-a5cd-fd08e1ccf1bb admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.367724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaa4df-f39e-4858-a5cd-fd08e1ccf1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.367984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaa4df-f39e-4858-a5cd-fd08e1ccf1bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.368651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aaa4df-f39e-4858-a5cd-fd08e1ccf1bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:56.370540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aaa4df-f39e-4858-a5cd-fd08e1ccf1bb admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:56.414978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1219/2439] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:21:56 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:56.415971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaa4df-f39e-4858-a5cd-fd08e1ccf1bb admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.417693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-36aaa4df-f39e-4858-a5cd-fd08e1ccf1bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:56.423055 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10740af5-a0b2-468b-b661-e300486ab9dd admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:56.443312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:56.474708 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc6773-f6b9-4a47-9b3c-ca0f3dea9df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:56.475321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc6773-f6b9-4a47-9b3c-ca0f3dea9df5 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.475587 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc6773-f6b9-4a47-9b3c-ca0f3dea9df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.475849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc6773-f6b9-4a47-9b3c-ca0f3dea9df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.477627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3bcc6773-f6b9-4a47-9b3c-ca0f3dea9df5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:56.488960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaa4df-f39e-4858-a5cd-fd08e1ccf1bb admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:56.494064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:56.494687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.494956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.495221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] PATH_INFO: `/v3/users/07ed99eb59b9478d98406d8e68aebbbd` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.496365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ed99eb59b9478d98406d8e68aebbbd)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:56.498293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:56.536612 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc6773-f6b9-4a47-9b3c-ca0f3dea9df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:56.538057 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc6773-f6b9-4a47-9b3c-ca0f3dea9df5 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:56.547638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1221/2440] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:56.548482 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc6773-f6b9-4a47-9b3c-ca0f3dea9df5 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.549785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3bcc6773-f6b9-4a47-9b3c-ca0f3dea9df5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:56.552078 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ed99eb59b9478d98406d8e68aebbbd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.560813 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ed99eb59b9478d98406d8e68aebbbd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.564495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc6773-f6b9-4a47-9b3c-ca0f3dea9df5 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:56.579783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ed99eb59b9478d98406d8e68aebbbd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.584838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744e11e-a5bc-4c36-8b7d-27f7fdc173b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:56.585641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744e11e-a5bc-4c36-8b7d-27f7fdc173b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.586040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744e11e-a5bc-4c36-8b7d-27f7fdc173b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.586378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744e11e-a5bc-4c36-8b7d-27f7fdc173b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.587430 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744e11e-a5bc-4c36-8b7d-27f7fdc173b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:56.589960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d744e11e-a5bc-4c36-8b7d-27f7fdc173b9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:56.599086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ed99eb59b9478d98406d8e68aebbbd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.599906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ed99eb59b9478d98406d8e68aebbbd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:56.608275 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1220/2441] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:21:56 2022] DELETE /identity/v3/users/07ed99eb59b9478d98406d8e68aebbbd => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:56.617554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1222/2442] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:21:56 2022] POST /identity/v3/users => generated 292 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:21:56.621577 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.623970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:56.676282 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:56.698494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f7dfb-843a-4956-b441-72fd3eff2da8 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:56.705150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a40b39-54e8-49b4-a711-f231d4be4819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:56.705661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a40b39-54e8-49b4-a711-f231d4be4819 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.705928 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a40b39-54e8-49b4-a711-f231d4be4819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.706191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a40b39-54e8-49b4-a711-f231d4be4819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.706834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a40b39-54e8-49b4-a711-f231d4be4819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:56.708428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a40b39-54e8-49b4-a711-f231d4be4819 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:56.720554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1221/2443] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:21:56.724568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744e11e-a5bc-4c36-8b7d-27f7fdc173b9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.737759 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744e11e-a5bc-4c36-8b7d-27f7fdc173b9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:56.752355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ae5c2-0a58-43af-a02b-18f7984fa70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:56.753159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ae5c2-0a58-43af-a02b-18f7984fa70b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.753552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ae5c2-0a58-43af-a02b-18f7984fa70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.754567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ae5c2-0a58-43af-a02b-18f7984fa70b admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.755586 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ae5c2-0a58-43af-a02b-18f7984fa70b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:56.759773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ae5c2-0a58-43af-a02b-18f7984fa70b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:56.783831 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1223/2444] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:21:56.784053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a40b39-54e8-49b4-a711-f231d4be4819 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.793374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a40b39-54e8-49b4-a711-f231d4be4819 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:56.801430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa6757-104a-401f-b26b-121ed1821cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:56.802390 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa6757-104a-401f-b26b-121ed1821cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.802620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa6757-104a-401f-b26b-121ed1821cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.802896 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa6757-104a-401f-b26b-121ed1821cd0 admin admin] PATH_INFO: `/v3/projects/1b76f9eee7a14eb581dcdc19101fc774/users/09d964117e9f449aaf3c768ccd395983/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.823765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa6757-104a-401f-b26b-121ed1821cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b76f9eee7a14eb581dcdc19101fc774, user_id=09d964117e9f449aaf3c768ccd395983, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:56.827391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa6757-104a-401f-b26b-121ed1821cd0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:56.843404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1222/2445] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:21:56 2022] PUT /identity/v3/projects/1b76f9eee7a14eb581dcdc19101fc774/users/09d964117e9f449aaf3c768ccd395983/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:21:56.848507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21affe90-b918-4987-9e6b-78371fe93566 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:56.848743 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21affe90-b918-4987-9e6b-78371fe93566 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:56.848945 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21affe90-b918-4987-9e6b-78371fe93566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:56.870972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-21affe90-b918-4987-9e6b-78371fe93566 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:21:56.872316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-21affe90-b918-4987-9e6b-78371fe93566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:56.971980 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1224/2446] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:21:56.973694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa6757-104a-401f-b26b-121ed1821cd0 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.975646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d0fa6757-104a-401f-b26b-121ed1821cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:56.985731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21affe90-b918-4987-9e6b-78371fe93566 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:21:56.987365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-21affe90-b918-4987-9e6b-78371fe93566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:21:57.011212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa6757-104a-401f-b26b-121ed1821cd0 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:57.024807 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:57.025463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:57.025731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:57.025997 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] PATH_INFO: `/v3/projects/9ff457e70fb94b64bc643104d79ef8f7` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:57.027142 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ff457e70fb94b64bc643104d79ef8f7)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:57.029108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:57.047514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21affe90-b918-4987-9e6b-78371fe93566 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:21:57.051313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae2c67-8320-4350-b370-57b49fafa3ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:21:57.051918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae2c67-8320-4350-b370-57b49fafa3ca service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:21:57.052186 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae2c67-8320-4350-b370-57b49fafa3ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:21:57.052445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae2c67-8320-4350-b370-57b49fafa3ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:21:57.058511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:21:57.063580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae2c67-8320-4350-b370-57b49fafa3ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:21:57.065525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae2c67-8320-4350-b370-57b49fafa3ca service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:21:57.076303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1225/2447] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:21:57.083340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ff457e70fb94b64bc643104d79ef8f7', 'request_id': 'req-2dc3e2f2-9881-44b9-b891-415770bf8a08'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:57.090184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ff457e70fb94b64bc643104d79ef8f7', 'request_id': 'req-2dc3e2f2-9881-44b9-b891-415770bf8a08'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:57.096035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ff457e70fb94b64bc643104d79ef8f7', 'request_id': 'req-2dc3e2f2-9881-44b9-b891-415770bf8a08'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:57.111118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] The token cache is being invalidate because project 9ff457e70fb94b64bc643104d79ef8f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:21:57.111404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:57.112033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:21:57.113801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1223/2448] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:21:56 2022] DELETE /identity/v3/projects/9ff457e70fb94b64bc643104d79ef8f7 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:01.306393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc6dfd-596f-4d30-afd7-bf0057f27546 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.306965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc6dfd-596f-4d30-afd7-bf0057f27546 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.306965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc6dfd-596f-4d30-afd7-bf0057f27546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.336059 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-7afc6dfd-596f-4d30-afd7-bf0057f27546 None None] MFA Rules not processed for user `09d964117e9f449aaf3c768ccd395983`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:01.337307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7afc6dfd-596f-4d30-afd7-bf0057f27546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:01.410110 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1226/2449] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:22:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:01.414688 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.416174 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:01.487914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3e2f2-9881-44b9-b891-415770bf8a08 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.492967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c85ea-a884-4094-aa67-e3bd47f8ac9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.493581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c85ea-a884-4094-aa67-e3bd47f8ac9d admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.493857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c85ea-a884-4094-aa67-e3bd47f8ac9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.494123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c85ea-a884-4094-aa67-e3bd47f8ac9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.494762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c85ea-a884-4094-aa67-e3bd47f8ac9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:01.496721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c85ea-a884-4094-aa67-e3bd47f8ac9d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:01.526942 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1224/2450] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:01 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:01.531451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc6dfd-596f-4d30-afd7-bf0057f27546 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.586004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc6dfd-596f-4d30-afd7-bf0057f27546 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.657002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb581bc-3bfa-4274-9171-7a243c4c8dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.657678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb581bc-3bfa-4274-9171-7a243c4c8dcb admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.657960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb581bc-3bfa-4274-9171-7a243c4c8dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.658256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb581bc-3bfa-4274-9171-7a243c4c8dcb admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.658912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb581bc-3bfa-4274-9171-7a243c4c8dcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:01.660941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb581bc-3bfa-4274-9171-7a243c4c8dcb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:01.685868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1227/2451] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:01 2022] POST /identity/v3/users => generated 298 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:01.693306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c85ea-a884-4094-aa67-e3bd47f8ac9d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.708170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c85ea-a884-4094-aa67-e3bd47f8ac9d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.712853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a69bd-b073-4996-a7ca-89d927db5d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.713469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a69bd-b073-4996-a7ca-89d927db5d95 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.713745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a69bd-b073-4996-a7ca-89d927db5d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.714039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a69bd-b073-4996-a7ca-89d927db5d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.714689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a69bd-b073-4996-a7ca-89d927db5d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:01.716457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a69bd-b073-4996-a7ca-89d927db5d95 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:01.727900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1225/2452] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:01.732157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb581bc-3bfa-4274-9171-7a243c4c8dcb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.743595 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb581bc-3bfa-4274-9171-7a243c4c8dcb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.748377 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c1193-9965-4374-8e82-89098fd6d01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.749155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c1193-9965-4374-8e82-89098fd6d01d admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.749616 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c1193-9965-4374-8e82-89098fd6d01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.750368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c1193-9965-4374-8e82-89098fd6d01d admin admin] PATH_INFO: `/v3/projects/ce06db306562401dafdefc49d5a58b36/users/947d3e1d6e994aacb03df034a23c42fd/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.768036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c1193-9965-4374-8e82-89098fd6d01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce06db306562401dafdefc49d5a58b36, user_id=947d3e1d6e994aacb03df034a23c42fd, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:01.770787 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c1193-9965-4374-8e82-89098fd6d01d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:01.785949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1228/2453] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:01 2022] PUT /identity/v3/projects/ce06db306562401dafdefc49d5a58b36/users/947d3e1d6e994aacb03df034a23c42fd/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:01.790198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a69bd-b073-4996-a7ca-89d927db5d95 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.805155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a69bd-b073-4996-a7ca-89d927db5d95 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.815172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:01.868537 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03e061-f638-4936-aab6-bb4e2838ffce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.869201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03e061-f638-4936-aab6-bb4e2838ffce admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.869476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03e061-f638-4936-aab6-bb4e2838ffce admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.869748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03e061-f638-4936-aab6-bb4e2838ffce admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.870405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03e061-f638-4936-aab6-bb4e2838ffce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:01.872015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03e061-f638-4936-aab6-bb4e2838ffce admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:01.883471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1226/2454] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:22:01.887868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c1193-9965-4374-8e82-89098fd6d01d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:01.898760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c1193-9965-4374-8e82-89098fd6d01d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:01.906555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e359c-e62f-4d70-86df-695629ab2db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:01.907235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e359c-e62f-4d70-86df-695629ab2db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:01.907537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e359c-e62f-4d70-86df-695629ab2db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:01.907910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e359c-e62f-4d70-86df-695629ab2db3 admin admin] PATH_INFO: `/v3/projects/ce06db306562401dafdefc49d5a58b36/users/947d3e1d6e994aacb03df034a23c42fd/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:01.909973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e359c-e62f-4d70-86df-695629ab2db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce06db306562401dafdefc49d5a58b36, user_id=947d3e1d6e994aacb03df034a23c42fd, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:01.912706 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e359c-e62f-4d70-86df-695629ab2db3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:01.924560 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1229/2455] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:01 2022] PUT /identity/v3/projects/ce06db306562401dafdefc49d5a58b36/users/947d3e1d6e994aacb03df034a23c42fd/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:07.550646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f355bc-2df9-48bc-87a9-c757da65d300 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:07.551070 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f355bc-2df9-48bc-87a9-c757da65d300 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:07.551252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f355bc-2df9-48bc-87a9-c757da65d300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:07.601063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e1f355bc-2df9-48bc-87a9-c757da65d300 None None] MFA Rules not processed for user `947d3e1d6e994aacb03df034a23c42fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:07.601703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e1f355bc-2df9-48bc-87a9-c757da65d300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:07.677750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1227/2456] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:22:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:07.690514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e359c-e62f-4d70-86df-695629ab2db3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:07.692290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-156e359c-e62f-4d70-86df-695629ab2db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:07.748134 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:07.770830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e359c-e62f-4d70-86df-695629ab2db3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:07.774476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff855e4f-7e0a-47d0-b290-e839da658975 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac783fbe95ea4b74bd46af415c9734fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:07.775089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff855e4f-7e0a-47d0-b290-e839da658975 service glance] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:07.775357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff855e4f-7e0a-47d0-b290-e839da658975 service glance] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:07.775754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff855e4f-7e0a-47d0-b290-e839da658975 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:07.835101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff855e4f-7e0a-47d0-b290-e839da658975 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:07.836547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff855e4f-7e0a-47d0-b290-e839da658975 service glance] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:07.846081 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1230/2457] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:08.000535 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e44d51-8091-46b5-a733-2ef0fc01bed9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.001605 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e44d51-8091-46b5-a733-2ef0fc01bed9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.001605 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e44d51-8091-46b5-a733-2ef0fc01bed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.021424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c5e44d51-8091-46b5-a733-2ef0fc01bed9 None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:08.022636 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c5e44d51-8091-46b5-a733-2ef0fc01bed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:08.082527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1228/2458] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:22:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:08.097650 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff855e4f-7e0a-47d0-b290-e839da658975 service glance] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:08.099258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ff855e4f-7e0a-47d0-b290-e839da658975 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:08.214592 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff855e4f-7e0a-47d0-b290-e839da658975 service glance] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:08.225223 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22653f-e564-462f-8b60-1e3796dc6094 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:08.225996 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22653f-e564-462f-8b60-1e3796dc6094 service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:08.226376 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22653f-e564-462f-8b60-1e3796dc6094 service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:08.226745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22653f-e564-462f-8b60-1e3796dc6094 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:08.237049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22653f-e564-462f-8b60-1e3796dc6094 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:08.238789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f22653f-e564-462f-8b60-1e3796dc6094 service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:08.248863 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1231/2459] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:22:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:22:17.442445 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21661c5-44ae-4ffe-bb9d-e0f04b449842 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:17.443025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21661c5-44ae-4ffe-bb9d-e0f04b449842 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:17.443025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21661c5-44ae-4ffe-bb9d-e0f04b449842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:17.464215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f21661c5-44ae-4ffe-bb9d-e0f04b449842 None None] MFA Rules not processed for user `09d964117e9f449aaf3c768ccd395983`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:17.465448 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f21661c5-44ae-4ffe-bb9d-e0f04b449842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:17.473075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1229/2460] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:22:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:18.244766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=64 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:22.586454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021d63f-4dc7-4309-9abb-b8152ae0d20b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:22.587056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021d63f-4dc7-4309-9abb-b8152ae0d20b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:22.587056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021d63f-4dc7-4309-9abb-b8152ae0d20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:22.607520 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d021d63f-4dc7-4309-9abb-b8152ae0d20b None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:22.608956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d021d63f-4dc7-4309-9abb-b8152ae0d20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:22.616905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1232/2461] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:22:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:22.629541 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21661c5-44ae-4ffe-bb9d-e0f04b449842 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:22.642075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21661c5-44ae-4ffe-bb9d-e0f04b449842 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:22.647170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f72dfd-28ed-4745-9cad-39f2f2ec7bf2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:22.647823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f72dfd-28ed-4745-9cad-39f2f2ec7bf2 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:22.648108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f72dfd-28ed-4745-9cad-39f2f2ec7bf2 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:22.648375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f72dfd-28ed-4745-9cad-39f2f2ec7bf2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:22.657485 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f72dfd-28ed-4745-9cad-39f2f2ec7bf2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:22.659088 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f72dfd-28ed-4745-9cad-39f2f2ec7bf2 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:22.669125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1230/2462] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:22:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:22:25.860549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021d63f-4dc7-4309-9abb-b8152ae0d20b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:25.862233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d021d63f-4dc7-4309-9abb-b8152ae0d20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:25.935951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021d63f-4dc7-4309-9abb-b8152ae0d20b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:25.941820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:25.942464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:25.942737 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:25.943004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] PATH_INFO: `/v3/users/a2567a5095b4423eb37803483977f8d4` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:25.944165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2567a5095b4423eb37803483977f8d4)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:25.946138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:26.004465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2567a5095b4423eb37803483977f8d4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:26.006394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2567a5095b4423eb37803483977f8d4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:26.015431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2567a5095b4423eb37803483977f8d4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:26.027531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2567a5095b4423eb37803483977f8d4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:26.028280 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2567a5095b4423eb37803483977f8d4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:26.043029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1233/2463] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:25 2022] DELETE /identity/v3/users/a2567a5095b4423eb37803483977f8d4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:26.047805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d066b62-a05c-4d7d-aa07-47f9803d7c6c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:26.048305 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d066b62-a05c-4d7d-aa07-47f9803d7c6c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:26.048425 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d066b62-a05c-4d7d-aa07-47f9803d7c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:26.067712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-1d066b62-a05c-4d7d-aa07-47f9803d7c6c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:26.068956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1d066b62-a05c-4d7d-aa07-47f9803d7c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:26.081920 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:26.128389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1231/2464] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:26.138556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:26.140242 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:26.235519 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199559f-d6e1-4cd1-b985-e68847637274 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:26.239819 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd0c8c-e305-4c4b-9c26-52cdddb82766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:26.240470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd0c8c-e305-4c4b-9c26-52cdddb82766 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:26.240772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd0c8c-e305-4c4b-9c26-52cdddb82766 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:26.241051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd0c8c-e305-4c4b-9c26-52cdddb82766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:26.251860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd0c8c-e305-4c4b-9c26-52cdddb82766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:26.253510 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bd0c8c-e305-4c4b-9c26-52cdddb82766 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:26.265254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1234/2465] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:26.608512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d066b62-a05c-4d7d-aa07-47f9803d7c6c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:26.610370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-1d066b62-a05c-4d7d-aa07-47f9803d7c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:26.625450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d066b62-a05c-4d7d-aa07-47f9803d7c6c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:26.631783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86c0a3-98e7-4f05-a320-49637925c25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:26.633089 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86c0a3-98e7-4f05-a320-49637925c25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:26.633518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86c0a3-98e7-4f05-a320-49637925c25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:26.634218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86c0a3-98e7-4f05-a320-49637925c25d admin admin] PATH_INFO: `/v3/projects/9a5275503f8746648f26bd0d9bef8610` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:26.635869 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86c0a3-98e7-4f05-a320-49637925c25d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a5275503f8746648f26bd0d9bef8610)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:26.638613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86c0a3-98e7-4f05-a320-49637925c25d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:26.697075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3a86c0a3-98e7-4f05-a320-49637925c25d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a5275503f8746648f26bd0d9bef8610', 'request_id': 'req-3a86c0a3-98e7-4f05-a320-49637925c25d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:26.705751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3a86c0a3-98e7-4f05-a320-49637925c25d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a5275503f8746648f26bd0d9bef8610', 'request_id': 'req-3a86c0a3-98e7-4f05-a320-49637925c25d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:26.711490 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3a86c0a3-98e7-4f05-a320-49637925c25d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a5275503f8746648f26bd0d9bef8610', 'request_id': 'req-3a86c0a3-98e7-4f05-a320-49637925c25d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:26.728463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3a86c0a3-98e7-4f05-a320-49637925c25d admin admin] The token cache is being invalidate because project 9a5275503f8746648f26bd0d9bef8610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:26.729008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3a86c0a3-98e7-4f05-a320-49637925c25d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:26.729982 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3a86c0a3-98e7-4f05-a320-49637925c25d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:26.731860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1232/2466] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:26 2022] DELETE /identity/v3/projects/9a5275503f8746648f26bd0d9bef8610 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:29.945363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd0c8c-e305-4c4b-9c26-52cdddb82766 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:29.947495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-36bd0c8c-e305-4c4b-9c26-52cdddb82766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:29.961740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd0c8c-e305-4c4b-9c26-52cdddb82766 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:29.974505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:30.029269 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:30.029822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:30.030118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:30.030383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] PATH_INFO: `/v3/users/78a175a45c6441be860bfbbb7599efda` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:30.031529 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a175a45c6441be860bfbbb7599efda)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:30.033744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:30.086421 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a175a45c6441be860bfbbb7599efda'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:30.087068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a175a45c6441be860bfbbb7599efda'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:30.094298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a175a45c6441be860bfbbb7599efda'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:30.106080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a175a45c6441be860bfbbb7599efda'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:30.106701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a175a45c6441be860bfbbb7599efda'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:30.121795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1235/2467] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:29 2022] DELETE /identity/v3/users/78a175a45c6441be860bfbbb7599efda => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:30.126709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e3beb-d8ec-4dc6-8f71-abbeba8386b3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:30.127238 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e3beb-d8ec-4dc6-8f71-abbeba8386b3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:30.127343 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e3beb-d8ec-4dc6-8f71-abbeba8386b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:30.146836 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-048e3beb-d8ec-4dc6-8f71-abbeba8386b3 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:30.148027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-048e3beb-d8ec-4dc6-8f71-abbeba8386b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:30.213328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1233/2468] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:30.223597 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:30.225233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:30.286755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca36ae8-bc90-49da-8d1e-1e3d8db69e7d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:30.290644 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4097a-ae01-4c46-99f8-62f9f4397f97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:30.291239 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4097a-ae01-4c46-99f8-62f9f4397f97 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:30.291515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4097a-ae01-4c46-99f8-62f9f4397f97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:30.291779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4097a-ae01-4c46-99f8-62f9f4397f97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:30.301591 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4097a-ae01-4c46-99f8-62f9f4397f97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:30.303018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4097a-ae01-4c46-99f8-62f9f4397f97 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:30.314343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1236/2469] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:22:30.648855 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e3beb-d8ec-4dc6-8f71-abbeba8386b3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:30.650395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-048e3beb-d8ec-4dc6-8f71-abbeba8386b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:30.663676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e3beb-d8ec-4dc6-8f71-abbeba8386b3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:30.668280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:30.672997 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:30.673430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:30.673876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3 admin admin] PATH_INFO: `/v3/projects/6f972d1448134d4a8a1b68a34e40f643` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:30.675555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f972d1448134d4a8a1b68a34e40f643)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:30.678190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:30.727379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:30.731082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f972d1448134d4a8a1b68a34e40f643', 'request_id': 'req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:30.737714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f972d1448134d4a8a1b68a34e40f643', 'request_id': 'req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:30.745024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f972d1448134d4a8a1b68a34e40f643', 'request_id': 'req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:30.769206 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3 admin admin] The token cache is being invalidate because project 6f972d1448134d4a8a1b68a34e40f643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:30.769598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:30.773229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3f56aa67-eeeb-4859-8a23-0c4fae2df2f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:30.775709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1234/2470] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:30 2022] DELETE /identity/v3/projects/6f972d1448134d4a8a1b68a34e40f643 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:30.781010 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f160d5-d9c9-420e-b090-6d209f573a95 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:30.781295 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f160d5-d9c9-420e-b090-6d209f573a95 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:30.781564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f160d5-d9c9-420e-b090-6d209f573a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:30.807720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a9f160d5-d9c9-420e-b090-6d209f573a95 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:30.808874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a9f160d5-d9c9-420e-b090-6d209f573a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:30.852328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:30.918691 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1237/2471] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:30.919129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf59c9-32b4-42a5-a0bc-9d9c92deaf97 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:30.919411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf59c9-32b4-42a5-a0bc-9d9c92deaf97 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:30.919681 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf59c9-32b4-42a5-a0bc-9d9c92deaf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:30.941839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-63cf59c9-32b4-42a5-a0bc-9d9c92deaf97 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:30.943053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-63cf59c9-32b4-42a5-a0bc-9d9c92deaf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:30.951423 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1235/2472] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:22:30.955948 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f160d5-d9c9-420e-b090-6d209f573a95 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:30.965761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f160d5-d9c9-420e-b090-6d209f573a95 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:30.970492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373825b-75bc-446b-87a4-f2e4dfc77eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:30.971091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373825b-75bc-446b-87a4-f2e4dfc77eba admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:30.971367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373825b-75bc-446b-87a4-f2e4dfc77eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:30.971635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373825b-75bc-446b-87a4-f2e4dfc77eba admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:30.972290 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373825b-75bc-446b-87a4-f2e4dfc77eba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:22:30.972603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373825b-75bc-446b-87a4-f2e4dfc77eba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:30.974230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373825b-75bc-446b-87a4-f2e4dfc77eba admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:30.985550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1238/2473] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:22:30.989994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf59c9-32b4-42a5-a0bc-9d9c92deaf97 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:30.999840 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf59c9-32b4-42a5-a0bc-9d9c92deaf97 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:31.005148 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76696c4-6ada-483d-b385-e7c4bec46e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:31.005832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76696c4-6ada-483d-b385-e7c4bec46e9d admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:31.006104 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76696c4-6ada-483d-b385-e7c4bec46e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:31.006381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76696c4-6ada-483d-b385-e7c4bec46e9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:31.007069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76696c4-6ada-483d-b385-e7c4bec46e9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:31.009187 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76696c4-6ada-483d-b385-e7c4bec46e9d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:31.045324 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1236/2474] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:22:30 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:22:31.049600 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373825b-75bc-446b-87a4-f2e4dfc77eba admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:31.060263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373825b-75bc-446b-87a4-f2e4dfc77eba admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:31.125205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a46435-f2d0-4437-a4d2-4eb5c11466e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:31.125822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a46435-f2d0-4437-a4d2-4eb5c11466e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:31.126192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a46435-f2d0-4437-a4d2-4eb5c11466e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:31.126463 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a46435-f2d0-4437-a4d2-4eb5c11466e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:31.127119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a46435-f2d0-4437-a4d2-4eb5c11466e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:31.129363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a46435-f2d0-4437-a4d2-4eb5c11466e3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:31.153694 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1239/2475] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:22:31 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:22:31.160430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76696c4-6ada-483d-b385-e7c4bec46e9d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:31.179034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76696c4-6ada-483d-b385-e7c4bec46e9d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:31.183686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a5951-e768-401a-8140-4dd03bb620ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:31.184291 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a5951-e768-401a-8140-4dd03bb620ab admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:31.184566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a5951-e768-401a-8140-4dd03bb620ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:31.184868 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a5951-e768-401a-8140-4dd03bb620ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:31.185508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a5951-e768-401a-8140-4dd03bb620ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:31.187106 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a5951-e768-401a-8140-4dd03bb620ab admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:31.197691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1237/2476] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:31.201957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a46435-f2d0-4437-a4d2-4eb5c11466e3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:31.213269 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a46435-f2d0-4437-a4d2-4eb5c11466e3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:31.221277 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02a3bc-bf31-4fd8-8a9f-2f92585cddf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:31.221875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02a3bc-bf31-4fd8-8a9f-2f92585cddf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:31.222146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02a3bc-bf31-4fd8-8a9f-2f92585cddf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:31.222406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02a3bc-bf31-4fd8-8a9f-2f92585cddf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:31.223048 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02a3bc-bf31-4fd8-8a9f-2f92585cddf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:31.224676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02a3bc-bf31-4fd8-8a9f-2f92585cddf7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:31.236036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1240/2477] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:22:31.240207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a5951-e768-401a-8140-4dd03bb620ab admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:31.252052 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a5951-e768-401a-8140-4dd03bb620ab admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:31.258226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70c28-ae5d-4e35-ab4f-4f5938a03888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:31.258935 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70c28-ae5d-4e35-ab4f-4f5938a03888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:31.259204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70c28-ae5d-4e35-ab4f-4f5938a03888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:31.259495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d70c28-ae5d-4e35-ab4f-4f5938a03888 admin admin] PATH_INFO: `/v3/projects/0f477bbd51fb41cb82ea8022e30b0274/users/837b6476bf4242f0aaf9f75c8b1fddb2/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:31.275687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d70c28-ae5d-4e35-ab4f-4f5938a03888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f477bbd51fb41cb82ea8022e30b0274, user_id=837b6476bf4242f0aaf9f75c8b1fddb2, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:31.278372 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d70c28-ae5d-4e35-ab4f-4f5938a03888 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:31.297151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1238/2478] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:22:31 2022] PUT /identity/v3/projects/0f477bbd51fb41cb82ea8022e30b0274/users/837b6476bf4242f0aaf9f75c8b1fddb2/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:31.312643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d6ffa-f36d-4075-b12e-0bf520a04474 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:31.312829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d6ffa-f36d-4075-b12e-0bf520a04474 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:31.313145 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d6ffa-f36d-4075-b12e-0bf520a04474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:31.342404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-672d6ffa-f36d-4075-b12e-0bf520a04474 None None] MFA Rules not processed for user `837b6476bf4242f0aaf9f75c8b1fddb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:31.343609 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-672d6ffa-f36d-4075-b12e-0bf520a04474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:31.441517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1241/2479] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:22:31 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:31.460793 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70c28-ae5d-4e35-ab4f-4f5938a03888 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:31.462598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-40d70c28-ae5d-4e35-ab4f-4f5938a03888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:31.522518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d70c28-ae5d-4e35-ab4f-4f5938a03888 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:31.526379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d763829-7184-4bc4-9473-4ec4fc360329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:31.526978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d763829-7184-4bc4-9473-4ec4fc360329 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:31.527248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d763829-7184-4bc4-9473-4ec4fc360329 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:31.527513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d763829-7184-4bc4-9473-4ec4fc360329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:31.536499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d763829-7184-4bc4-9473-4ec4fc360329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:31.537957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d763829-7184-4bc4-9473-4ec4fc360329 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:31.549267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1239/2480] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:31 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:41.437788 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:41.543419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=63,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:43.955247 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d6ffa-f36d-4075-b12e-0bf520a04474 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:44.020008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d6ffa-f36d-4075-b12e-0bf520a04474 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:44.024972 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:44.025606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:44.025944 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:44.026215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] PATH_INFO: `/v3/users/837b6476bf4242f0aaf9f75c8b1fddb2` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:44.027360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] RBAC: Authorizing `identity:delete_user(user_id=837b6476bf4242f0aaf9f75c8b1fddb2)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:44.029327 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:44.078476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837b6476bf4242f0aaf9f75c8b1fddb2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:44.079161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837b6476bf4242f0aaf9f75c8b1fddb2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:44.086508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837b6476bf4242f0aaf9f75c8b1fddb2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:44.098202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837b6476bf4242f0aaf9f75c8b1fddb2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:44.098817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837b6476bf4242f0aaf9f75c8b1fddb2'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:44.113040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1242/2481] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:22:43 2022] DELETE /identity/v3/users/837b6476bf4242f0aaf9f75c8b1fddb2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:44.118575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f264c46-4cd9-4092-85ad-e8ad02a1bc85 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:44.119073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f264c46-4cd9-4092-85ad-e8ad02a1bc85 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:44.119073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f264c46-4cd9-4092-85ad-e8ad02a1bc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:44.138397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4f264c46-4cd9-4092-85ad-e8ad02a1bc85 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:22:44.139591 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4f264c46-4cd9-4092-85ad-e8ad02a1bc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:44.200240 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1240/2482] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:22:44.211121 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:44.213087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:44.274606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49338cbf-2ef7-4d32-bfb9-4f4e1717330b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:44.278755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed31dcf-42d0-4f17-a10f-fda2fef03dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:44.279350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed31dcf-42d0-4f17-a10f-fda2fef03dbf service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:44.279621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed31dcf-42d0-4f17-a10f-fda2fef03dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:44.279873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed31dcf-42d0-4f17-a10f-fda2fef03dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:44.289462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed31dcf-42d0-4f17-a10f-fda2fef03dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:44.292777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed31dcf-42d0-4f17-a10f-fda2fef03dbf service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:44.302094 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1243/2483] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:22:44.581961 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f264c46-4cd9-4092-85ad-e8ad02a1bc85 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:44.583694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4f264c46-4cd9-4092-85ad-e8ad02a1bc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:44.596498 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f264c46-4cd9-4092-85ad-e8ad02a1bc85 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:44.601798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:44.602438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:44.602736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:44.602990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] PATH_INFO: `/v3/projects/0f477bbd51fb41cb82ea8022e30b0274` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:44.604163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f477bbd51fb41cb82ea8022e30b0274)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:22:44.606093 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:22:44.656415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f477bbd51fb41cb82ea8022e30b0274', 'request_id': 'req-80e9e4ad-7fc5-4a02-8c23-18627f89919e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:44.663177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f477bbd51fb41cb82ea8022e30b0274', 'request_id': 'req-80e9e4ad-7fc5-4a02-8c23-18627f89919e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:44.669367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f477bbd51fb41cb82ea8022e30b0274', 'request_id': 'req-80e9e4ad-7fc5-4a02-8c23-18627f89919e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:44.685926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] The token cache is being invalidate because project 0f477bbd51fb41cb82ea8022e30b0274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:22:44.686216 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:44.686910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:22:44.688532 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1241/2484] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:22:44 2022] DELETE /identity/v3/projects/0f477bbd51fb41cb82ea8022e30b0274 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:22:54.298032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:54.683033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:22:59.922597 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed31dcf-42d0-4f17-a10f-fda2fef03dbf service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:22:59.924232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7ed31dcf-42d0-4f17-a10f-fda2fef03dbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:22:59.940471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed31dcf-42d0-4f17-a10f-fda2fef03dbf service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:22:59.996645 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064ceca-0aeb-4707-b769-7332bfec42cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:22:59.997254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064ceca-0aeb-4707-b769-7332bfec42cd service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:22:59.997528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064ceca-0aeb-4707-b769-7332bfec42cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:22:59.997795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064ceca-0aeb-4707-b769-7332bfec42cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:22:59.999689 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8064ceca-0aeb-4707-b769-7332bfec42cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:00.076379 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064ceca-0aeb-4707-b769-7332bfec42cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:00.077856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064ceca-0aeb-4707-b769-7332bfec42cd service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:00.088717 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1244/2485] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:22:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:01.307529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:01.309803 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:01.323526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9e4ad-7fc5-4a02-8c23-18627f89919e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:01.327994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedaea80-9c70-45c8-90a3-cb676f77e326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:01.328615 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedaea80-9c70-45c8-90a3-cb676f77e326 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:01.328894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedaea80-9c70-45c8-90a3-cb676f77e326 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:01.329165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedaea80-9c70-45c8-90a3-cb676f77e326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:01.330986 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dedaea80-9c70-45c8-90a3-cb676f77e326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:01.345391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedaea80-9c70-45c8-90a3-cb676f77e326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:01.347460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedaea80-9c70-45c8-90a3-cb676f77e326 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:01.359246 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1242/2486] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:10.083884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=54 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:10.237905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064ceca-0aeb-4707-b769-7332bfec42cd service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:10.239610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8064ceca-0aeb-4707-b769-7332bfec42cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:10.253262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064ceca-0aeb-4707-b769-7332bfec42cd service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:10.257697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f2cfe-3f42-480c-9e24-5436c923f8b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:10.258291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f2cfe-3f42-480c-9e24-5436c923f8b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:10.258564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f2cfe-3f42-480c-9e24-5436c923f8b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:10.258839 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f2cfe-3f42-480c-9e24-5436c923f8b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:10.261373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d50f2cfe-3f42-480c-9e24-5436c923f8b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:10.328481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f2cfe-3f42-480c-9e24-5436c923f8b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:10.329967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f2cfe-3f42-480c-9e24-5436c923f8b5 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:10.340208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1245/2487] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:23:11.353598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:12.202143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedaea80-9c70-45c8-90a3-cb676f77e326 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:12.203867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dedaea80-9c70-45c8-90a3-cb676f77e326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:12.220021 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedaea80-9c70-45c8-90a3-cb676f77e326 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:12.227325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b519ecf2-e078-4650-9fc9-917b5e2ae00b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:12.227955 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519ecf2-e078-4650-9fc9-917b5e2ae00b service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:12.228234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519ecf2-e078-4650-9fc9-917b5e2ae00b service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:12.228504 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519ecf2-e078-4650-9fc9-917b5e2ae00b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:12.231095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b519ecf2-e078-4650-9fc9-917b5e2ae00b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:12.299160 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b519ecf2-e078-4650-9fc9-917b5e2ae00b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:12.300778 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b519ecf2-e078-4650-9fc9-917b5e2ae00b service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:12.315262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1243/2488] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:16.227812 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f2cfe-3f42-480c-9e24-5436c923f8b5 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:16.242532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f2cfe-3f42-480c-9e24-5436c923f8b5 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:16.248561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222dbeb1-d42a-470d-a09a-e99d573bb3a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:16.249188 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222dbeb1-d42a-470d-a09a-e99d573bb3a4 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:16.249459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222dbeb1-d42a-470d-a09a-e99d573bb3a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:16.249765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222dbeb1-d42a-470d-a09a-e99d573bb3a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:16.251612 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-222dbeb1-d42a-470d-a09a-e99d573bb3a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:16.313566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222dbeb1-d42a-470d-a09a-e99d573bb3a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:16.315264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222dbeb1-d42a-470d-a09a-e99d573bb3a4 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:16.325011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1246/2489] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:21.660728 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38291-4ff3-4896-85d1-5a9e9c2abb4e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:21.661215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38291-4ff3-4896-85d1-5a9e9c2abb4e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:21.661333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a38291-4ff3-4896-85d1-5a9e9c2abb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:21.688580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a6a38291-4ff3-4896-85d1-5a9e9c2abb4e None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:21.690376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a6a38291-4ff3-4896-85d1-5a9e9c2abb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:21.749745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1244/2490] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:23:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:21.774697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222dbeb1-d42a-470d-a09a-e99d573bb3a4 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:21.776228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-222dbeb1-d42a-470d-a09a-e99d573bb3a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:21.858242 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222dbeb1-d42a-470d-a09a-e99d573bb3a4 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:21.864304 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd357b9d-e4c1-4487-9fb4-7090f5accb0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:21.864953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd357b9d-e4c1-4487-9fb4-7090f5accb0d service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:21.865501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd357b9d-e4c1-4487-9fb4-7090f5accb0d service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:21.865501 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd357b9d-e4c1-4487-9fb4-7090f5accb0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:21.874879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd357b9d-e4c1-4487-9fb4-7090f5accb0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:21.875997 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd357b9d-e4c1-4487-9fb4-7090f5accb0d service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:21.884676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1247/2491] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:23:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:23:31.742049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=53 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:31.881597 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:32.810189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a38291-4ff3-4896-85d1-5a9e9c2abb4e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:32.820511 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a38291-4ff3-4896-85d1-5a9e9c2abb4e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:32.824323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d345a9-f9ec-4dc4-8528-041fdce9b5de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:32.824965 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d345a9-f9ec-4dc4-8528-041fdce9b5de service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:32.825726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d345a9-f9ec-4dc4-8528-041fdce9b5de service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:32.825726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d345a9-f9ec-4dc4-8528-041fdce9b5de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:32.827297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c7d345a9-f9ec-4dc4-8528-041fdce9b5de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:32.837581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d345a9-f9ec-4dc4-8528-041fdce9b5de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:32.839005 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d345a9-f9ec-4dc4-8528-041fdce9b5de service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:32.847698 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1245/2492] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:37.048298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd357b9d-e4c1-4487-9fb4-7090f5accb0d service swift] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:37.058014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd357b9d-e4c1-4487-9fb4-7090f5accb0d service swift] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:37.062238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ac4bc-0b47-42b2-8608-a185889045fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:37.062821 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ac4bc-0b47-42b2-8608-a185889045fd service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:37.063086 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ac4bc-0b47-42b2-8608-a185889045fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:37.063346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ac4bc-0b47-42b2-8608-a185889045fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:37.065340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e78ac4bc-0b47-42b2-8608-a185889045fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:37.074998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ac4bc-0b47-42b2-8608-a185889045fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:37.076462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ac4bc-0b47-42b2-8608-a185889045fd service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:37.084830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1248/2493] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:37.221875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3642abd-2e01-4110-8d4f-70ac87405b69 None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:37.222446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3642abd-2e01-4110-8d4f-70ac87405b69 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:37.222446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3642abd-2e01-4110-8d4f-70ac87405b69 None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:37.223995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1246/2494] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:23:37.230350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0c086-2764-4900-88d7-a1055185b9fd None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:37.230634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0c086-2764-4900-88d7-a1055185b9fd None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:37.230904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0c086-2764-4900-88d7-a1055185b9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:37.252854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-86d0c086-2764-4900-88d7-a1055185b9fd None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:37.254124 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-86d0c086-2764-4900-88d7-a1055185b9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:37.263886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1249/2495] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:23:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:37.276441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3642abd-2e01-4110-8d4f-70ac87405b69 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:37.278293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b3642abd-2e01-4110-8d4f-70ac87405b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:37.338326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3642abd-2e01-4110-8d4f-70ac87405b69 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:37.342417 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde1bb8-cdb3-4cc4-b39b-3c6fb5f216d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac783fbe95ea4b74bd46af415c9734fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:37.343024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde1bb8-cdb3-4cc4-b39b-3c6fb5f216d0 service glance] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:37.343290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde1bb8-cdb3-4cc4-b39b-3c6fb5f216d0 service glance] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:37.343552 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde1bb8-cdb3-4cc4-b39b-3c6fb5f216d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:37.354184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde1bb8-cdb3-4cc4-b39b-3c6fb5f216d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:37.355622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde1bb8-cdb3-4cc4-b39b-3c6fb5f216d0 service glance] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:37.366596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1247/2496] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:37.441745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db913d-3f2b-4e75-88cc-d89e6c633204 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:37.441966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db913d-3f2b-4e75-88cc-d89e6c633204 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:37.442185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db913d-3f2b-4e75-88cc-d89e6c633204 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:37.443542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1250/2497] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:23:37.449184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bce9e-7822-4e44-914e-8d24b9233927 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:37.449573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bce9e-7822-4e44-914e-8d24b9233927 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:37.449984 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bce9e-7822-4e44-914e-8d24b9233927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:37.473495 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-654bce9e-7822-4e44-914e-8d24b9233927 None None] MFA Rules not processed for user `02afdedf8d394bbfa977ab7a22fafea7`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:37.474686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-654bce9e-7822-4e44-914e-8d24b9233927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:37.482371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1248/2498] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:23:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:37.495390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db913d-3f2b-4e75-88cc-d89e6c633204 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:37.507108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db913d-3f2b-4e75-88cc-d89e6c633204 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:37.510923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d594e-cf59-40b6-bf05-038d691ec2ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:37.511492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d594e-cf59-40b6-bf05-038d691ec2ae service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:37.511777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d594e-cf59-40b6-bf05-038d691ec2ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:37.512042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d594e-cf59-40b6-bf05-038d691ec2ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:37.519335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d594e-cf59-40b6-bf05-038d691ec2ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:37.521260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d594e-cf59-40b6-bf05-038d691ec2ae service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:37.529032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1251/2499] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:37.621781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa061d0-9e01-4e07-81a7-d91b99cccd3f None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:37.622035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa061d0-9e01-4e07-81a7-d91b99cccd3f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:37.622333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa061d0-9e01-4e07-81a7-d91b99cccd3f None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:37.623724 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1249/2500] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:23:37.629106 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f082b6-92fc-4ea0-ad78-83fd614deabb None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:37.629387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f082b6-92fc-4ea0-ad78-83fd614deabb None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:37.629655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f082b6-92fc-4ea0-ad78-83fd614deabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:37.650866 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-68f082b6-92fc-4ea0-ad78-83fd614deabb None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:37.652082 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-68f082b6-92fc-4ea0-ad78-83fd614deabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:37.662822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1252/2501] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:23:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:37.673945 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa061d0-9e01-4e07-81a7-d91b99cccd3f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:37.687427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa061d0-9e01-4e07-81a7-d91b99cccd3f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:37.692125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89488e5f-21f9-457d-9600-292332cabfdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:37.692681 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89488e5f-21f9-457d-9600-292332cabfdc service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:37.692948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89488e5f-21f9-457d-9600-292332cabfdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:37.693215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89488e5f-21f9-457d-9600-292332cabfdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:37.704381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89488e5f-21f9-457d-9600-292332cabfdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:37.705815 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89488e5f-21f9-457d-9600-292332cabfdc service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:37.716942 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1250/2502] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:37.787660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f082b6-92fc-4ea0-ad78-83fd614deabb None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:37.789348 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-68f082b6-92fc-4ea0-ad78-83fd614deabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:37.863891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f082b6-92fc-4ea0-ad78-83fd614deabb None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:37.870930 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729270eb-f967-40bd-a998-05240aac8b32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:37.871533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729270eb-f967-40bd-a998-05240aac8b32 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:37.871845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729270eb-f967-40bd-a998-05240aac8b32 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:37.872098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729270eb-f967-40bd-a998-05240aac8b32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:37.874149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-729270eb-f967-40bd-a998-05240aac8b32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:37.886893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729270eb-f967-40bd-a998-05240aac8b32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:37.888489 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729270eb-f967-40bd-a998-05240aac8b32 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:37.898687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1253/2503] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:23:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:23:40.890587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1d44b-a5fc-4568-b636-4d81b5bb78fc None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:40.891108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1d44b-a5fc-4568-b636-4d81b5bb78fc None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:40.891108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1d44b-a5fc-4568-b636-4d81b5bb78fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:40.910664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0ac1d44b-a5fc-4568-b636-4d81b5bb78fc None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:23:40.911956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0ac1d44b-a5fc-4568-b636-4d81b5bb78fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:23:40.919116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1251/2504] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:23:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:23:40.932369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729270eb-f967-40bd-a998-05240aac8b32 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:23:40.943932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729270eb-f967-40bd-a998-05240aac8b32 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:23:40.950012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a29f14-b57c-43a4-ae57-46eb0f9b3dec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:23:40.950606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a29f14-b57c-43a4-ae57-46eb0f9b3dec service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:23:40.950879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a29f14-b57c-43a4-ae57-46eb0f9b3dec service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:23:40.951152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a29f14-b57c-43a4-ae57-46eb0f9b3dec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:23:40.959439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a29f14-b57c-43a4-ae57-46eb0f9b3dec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:23:40.961041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a29f14-b57c-43a4-ae57-46eb0f9b3dec service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:23:40.971078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1254/2505] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:23:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:23:50.905826 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=44 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:23:50.965808 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=56 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:24:05.678888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1d44b-a5fc-4568-b636-4d81b5bb78fc None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:24:05.680348 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0ac1d44b-a5fc-4568-b636-4d81b5bb78fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:24:05.699493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1d44b-a5fc-4568-b636-4d81b5bb78fc None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:24:05.706708 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8d3cb-75d6-4a5a-a336-0d93117032ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:24:05.707332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8d3cb-75d6-4a5a-a336-0d93117032ef service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:05.707602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8d3cb-75d6-4a5a-a336-0d93117032ef service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:05.707865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8d3cb-75d6-4a5a-a336-0d93117032ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:05.709642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f4e8d3cb-75d6-4a5a-a336-0d93117032ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:24:05.772265 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8d3cb-75d6-4a5a-a336-0d93117032ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:24:05.773898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8d3cb-75d6-4a5a-a336-0d93117032ef service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:24:05.783701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1252/2506] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:24:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:24:07.738146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a29f14-b57c-43a4-ae57-46eb0f9b3dec service swift] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:24:07.754566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a29f14-b57c-43a4-ae57-46eb0f9b3dec service swift] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:24:07.760531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702b3ad-b868-4bdb-b0a7-04c45ef9ec7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:24:07.761163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702b3ad-b868-4bdb-b0a7-04c45ef9ec7c service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:07.761433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702b3ad-b868-4bdb-b0a7-04c45ef9ec7c service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:07.761715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702b3ad-b868-4bdb-b0a7-04c45ef9ec7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:07.763554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b702b3ad-b868-4bdb-b0a7-04c45ef9ec7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:24:07.776118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702b3ad-b868-4bdb-b0a7-04c45ef9ec7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:24:07.777752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702b3ad-b868-4bdb-b0a7-04c45ef9ec7c service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:24:07.788570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1255/2507] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:24:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:24:10.857756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00632925-b964-41f3-8aaf-aaa85a047d05 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:10.858270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00632925-b964-41f3-8aaf-aaa85a047d05 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:10.858365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00632925-b964-41f3-8aaf-aaa85a047d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:10.879315 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-00632925-b964-41f3-8aaf-aaa85a047d05 None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:24:10.880519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-00632925-b964-41f3-8aaf-aaa85a047d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:24:10.887555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1253/2508] 10.222.0.75 () {58 vars in 967 bytes} [Fri Sep 23 15:24:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:24:10.900329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702b3ad-b868-4bdb-b0a7-04c45ef9ec7c service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:24:10.912112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702b3ad-b868-4bdb-b0a7-04c45ef9ec7c service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:24:10.917632 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894be66b-c63c-45d6-b154-e2b9d662e2a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:24:10.918293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894be66b-c63c-45d6-b154-e2b9d662e2a3 service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:10.918625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894be66b-c63c-45d6-b154-e2b9d662e2a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:10.918892 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894be66b-c63c-45d6-b154-e2b9d662e2a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:10.927417 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894be66b-c63c-45d6-b154-e2b9d662e2a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:24:10.929013 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894be66b-c63c-45d6-b154-e2b9d662e2a3 service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:24:10.937183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1256/2509] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:24:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 15:24:20.873671 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=31 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:24:20.934168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=12 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:24:40.634398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af8440-745c-42d8-bcf1-13b8e4d43938 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:40.634971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af8440-745c-42d8-bcf1-13b8e4d43938 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:40.635199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af8440-745c-42d8-bcf1-13b8e4d43938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:40.657625 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-49af8440-745c-42d8-bcf1-13b8e4d43938 None None] MFA Rules not processed for user `4cfac62e1c304167abb971815d5da358`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:24:40.659193 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-49af8440-745c-42d8-bcf1-13b8e4d43938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:24:40.667283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1254/2510] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:24:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:24:47.177951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894be66b-c63c-45d6-b154-e2b9d662e2a3 service swift] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:24:47.202670 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894be66b-c63c-45d6-b154-e2b9d662e2a3 service swift] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:24:47.217215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40165b36-7783-4f99-9fff-d8f54fb8bb44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:24:47.218540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40165b36-7783-4f99-9fff-d8f54fb8bb44 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:47.218540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40165b36-7783-4f99-9fff-d8f54fb8bb44 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:47.218792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40165b36-7783-4f99-9fff-d8f54fb8bb44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:47.220753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-40165b36-7783-4f99-9fff-d8f54fb8bb44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:24:47.234249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40165b36-7783-4f99-9fff-d8f54fb8bb44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:24:47.235971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40165b36-7783-4f99-9fff-d8f54fb8bb44 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:24:47.248717 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1257/2511] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:24:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:24:50.650514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:24:54.547103 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af8440-745c-42d8-bcf1-13b8e4d43938 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:24:54.557657 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af8440-745c-42d8-bcf1-13b8e4d43938 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:24:54.561357 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa380abf-cde0-4b92-ac00-26912a1d3b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:24:54.562244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa380abf-cde0-4b92-ac00-26912a1d3b88 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:54.562576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa380abf-cde0-4b92-ac00-26912a1d3b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:54.562839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa380abf-cde0-4b92-ac00-26912a1d3b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:54.564772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fa380abf-cde0-4b92-ac00-26912a1d3b88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:24:54.575862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa380abf-cde0-4b92-ac00-26912a1d3b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:24:54.577404 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa380abf-cde0-4b92-ac00-26912a1d3b88 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:24:54.587730 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1255/2512] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:24:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:24:54.719219 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fc0ce-9f2d-47a6-bd72-fb415e6b75c9 None None] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:54.720315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fc0ce-9f2d-47a6-bd72-fb415e6b75c9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:54.720817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fc0ce-9f2d-47a6-bd72-fb415e6b75c9 None None] PATH_INFO: `/` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:54.722445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1258/2513] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:24:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:24:54.729338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2148e76-6119-4c04-876b-06b6496c83fa None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:54.729705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2148e76-6119-4c04-876b-06b6496c83fa None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:54.730056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2148e76-6119-4c04-876b-06b6496c83fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:54.752199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f2148e76-6119-4c04-876b-06b6496c83fa None None] MFA Rules not processed for user `02afdedf8d394bbfa977ab7a22fafea7`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:24:54.753596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f2148e76-6119-4c04-876b-06b6496c83fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:24:54.761688 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1256/2514] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:24:54.773939 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fc0ce-9f2d-47a6-bd72-fb415e6b75c9 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:24:54.785712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fc0ce-9f2d-47a6-bd72-fb415e6b75c9 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:24:54.789431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98daa7ad-a94b-430b-afd7-47c963483ad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:24:54.790018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daa7ad-a94b-430b-afd7-47c963483ad8 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:54.790298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daa7ad-a94b-430b-afd7-47c963483ad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:54.790549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daa7ad-a94b-430b-afd7-47c963483ad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:54.798511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98daa7ad-a94b-430b-afd7-47c963483ad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:24:54.799926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98daa7ad-a94b-430b-afd7-47c963483ad8 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:24:54.808524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1259/2515] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:24:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:24:54.890225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79931f-6ee7-45b0-9301-90973476f53f None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:54.890501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79931f-6ee7-45b0-9301-90973476f53f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:54.890774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79931f-6ee7-45b0-9301-90973476f53f None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:54.892123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1257/2516] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:24:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:24:54.897444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2de34e-9146-4638-a4ce-c59c9b50f28a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:54.897722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2de34e-9146-4638-a4ce-c59c9b50f28a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:54.897989 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2de34e-9146-4638-a4ce-c59c9b50f28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:54.918157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4b2de34e-9146-4638-a4ce-c59c9b50f28a None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:24:54.919352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4b2de34e-9146-4638-a4ce-c59c9b50f28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:24:54.929258 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1260/2517] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:24:54.939992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79931f-6ee7-45b0-9301-90973476f53f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:24:54.953441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79931f-6ee7-45b0-9301-90973476f53f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:24:54.957596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5b0a2-8d10-49d7-8915-9bab238fffc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:24:54.958188 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5b0a2-8d10-49d7-8915-9bab238fffc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:24:54.958457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5b0a2-8d10-49d7-8915-9bab238fffc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:24:54.958725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5b0a2-8d10-49d7-8915-9bab238fffc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:24:54.969444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5b0a2-8d10-49d7-8915-9bab238fffc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:24:54.970895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5b0a2-8d10-49d7-8915-9bab238fffc2 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:24:54.983255 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1258/2518] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:24:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:25:04.913460 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=15 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:25:04.977242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=15 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:25:28.884514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7c1e5-8581-4896-b830-f2c03bb4acac None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:25:28.885119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7c1e5-8581-4896-b830-f2c03bb4acac None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:25:28.885119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7c1e5-8581-4896-b830-f2c03bb4acac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:25:28.905315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-3bf7c1e5-8581-4896-b830-f2c03bb4acac None None] MFA Rules not processed for user `e38b853b47b64567b00092eb32201b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:25:28.906544 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3bf7c1e5-8581-4896-b830-f2c03bb4acac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:25:28.913817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1261/2519] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:25:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:25:38.899874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:02.879189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5b0a2-8d10-49d7-8915-9bab238fffc2 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:02.980267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5b0a2-8d10-49d7-8915-9bab238fffc2 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:02.986377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21432a70-e6e1-4a11-a9e5-ad36cecfa16d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:02.987056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21432a70-e6e1-4a11-a9e5-ad36cecfa16d service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:02.987467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21432a70-e6e1-4a11-a9e5-ad36cecfa16d service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:02.987750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21432a70-e6e1-4a11-a9e5-ad36cecfa16d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:02.989732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-21432a70-e6e1-4a11-a9e5-ad36cecfa16d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:03.038295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21432a70-e6e1-4a11-a9e5-ad36cecfa16d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:03.040530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21432a70-e6e1-4a11-a9e5-ad36cecfa16d service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:03.058919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1259/2520] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:05.489833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a1e44-9a64-4667-badf-a2b8d55b3a86 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:05.490351 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a1e44-9a64-4667-badf-a2b8d55b3a86 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:05.490351 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a1e44-9a64-4667-badf-a2b8d55b3a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:05.521731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-192a1e44-9a64-4667-badf-a2b8d55b3a86 None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:05.522978 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-192a1e44-9a64-4667-badf-a2b8d55b3a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:05.531184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1262/2521] 10.222.0.75 () {58 vars in 967 bytes} [Fri Sep 23 15:26:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:05.597956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21432a70-e6e1-4a11-a9e5-ad36cecfa16d service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:05.610325 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21432a70-e6e1-4a11-a9e5-ad36cecfa16d service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:05.618262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093e2a8-7872-4868-b15e-e4e70f38296c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:05.619017 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093e2a8-7872-4868-b15e-e4e70f38296c service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:05.619458 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093e2a8-7872-4868-b15e-e4e70f38296c service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:05.634629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093e2a8-7872-4868-b15e-e4e70f38296c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:05.648539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093e2a8-7872-4868-b15e-e4e70f38296c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:05.650551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093e2a8-7872-4868-b15e-e4e70f38296c service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:05.665066 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1260/2522] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:26:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:26:10.233340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9c63f-c4b5-42cb-911a-236569692291 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:10.233923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9c63f-c4b5-42cb-911a-236569692291 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:10.233923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9c63f-c4b5-42cb-911a-236569692291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:10.262967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-57e9c63f-c4b5-42cb-911a-236569692291 None None] MFA Rules not processed for user `4bded8b580f34e08b2890eb3430f4b80`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:10.264903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-57e9c63f-c4b5-42cb-911a-236569692291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:10.277345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1263/2523] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:11.823270 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093e2a8-7872-4868-b15e-e4e70f38296c service swift] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:11.865415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093e2a8-7872-4868-b15e-e4e70f38296c service swift] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:11.877837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc608403-0088-47f2-a46f-d7d905e64d8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:11.878723 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc608403-0088-47f2-a46f-d7d905e64d8b service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:11.879057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc608403-0088-47f2-a46f-d7d905e64d8b service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:11.879387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc608403-0088-47f2-a46f-d7d905e64d8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:11.884006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-bc608403-0088-47f2-a46f-d7d905e64d8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:11.918432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9c63f-c4b5-42cb-911a-236569692291 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:11.920282 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc608403-0088-47f2-a46f-d7d905e64d8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:11.921024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-57e9c63f-c4b5-42cb-911a-236569692291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:11.922395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc608403-0088-47f2-a46f-d7d905e64d8b service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:11.933969 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1261/2524] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:11.938980 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9c63f-c4b5-42cb-911a-236569692291 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:11.945658 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a0205-a6a0-42d4-a828-4e109fbd4547 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:11.947281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a0205-a6a0-42d4-a828-4e109fbd4547 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:11.947987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a0205-a6a0-42d4-a828-4e109fbd4547 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:11.948662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a0205-a6a0-42d4-a828-4e109fbd4547 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:11.960454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a0205-a6a0-42d4-a828-4e109fbd4547 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:11.962357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a0205-a6a0-42d4-a828-4e109fbd4547 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:11.973372 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1264/2525] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:15.397891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc608403-0088-47f2-a46f-d7d905e64d8b service placement] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:15.409077 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc608403-0088-47f2-a46f-d7d905e64d8b service placement] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:15.414494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7930d-7c75-4820-a611-c01d19df7ddc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:15.415176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7930d-7c75-4820-a611-c01d19df7ddc service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:15.415514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7930d-7c75-4820-a611-c01d19df7ddc service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:15.415880 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7930d-7c75-4820-a611-c01d19df7ddc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:15.418143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e9f7930d-7c75-4820-a611-c01d19df7ddc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:15.430004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7930d-7c75-4820-a611-c01d19df7ddc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:15.431517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7930d-7c75-4820-a611-c01d19df7ddc service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:15.443971 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1262/2526] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:15.461745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a0205-a6a0-42d4-a828-4e109fbd4547 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:15.463379 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ad7a0205-a6a0-42d4-a828-4e109fbd4547 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:15.579353 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a0205-a6a0-42d4-a828-4e109fbd4547 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:15.585073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:15.585704 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:15.585975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:15.586249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] PATH_INFO: `/v3/users/09d964117e9f449aaf3c768ccd395983` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:15.587936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=09d964117e9f449aaf3c768ccd395983)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:15.590868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:15.652264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09d964117e9f449aaf3c768ccd395983'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:15.652910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09d964117e9f449aaf3c768ccd395983'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:15.660216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09d964117e9f449aaf3c768ccd395983'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:15.672997 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09d964117e9f449aaf3c768ccd395983'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:15.673733 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09d964117e9f449aaf3c768ccd395983'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:15.688899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1265/2527] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:15 2022] DELETE /identity/v3/users/09d964117e9f449aaf3c768ccd395983 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:15.871751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7930d-7c75-4820-a611-c01d19df7ddc service placement] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:15.882392 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7930d-7c75-4820-a611-c01d19df7ddc service placement] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:15.946262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:15.946891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:15.947159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:15.947554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] PATH_INFO: `/v3/projects/1b76f9eee7a14eb581dcdc19101fc774` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:15.948726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b76f9eee7a14eb581dcdc19101fc774)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:15.950606 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:15.967589 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:16.000288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b76f9eee7a14eb581dcdc19101fc774', 'request_id': 'req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.006820 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b76f9eee7a14eb581dcdc19101fc774', 'request_id': 'req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.013580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b76f9eee7a14eb581dcdc19101fc774', 'request_id': 'req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.028193 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] The token cache is being invalidate because project 1b76f9eee7a14eb581dcdc19101fc774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:16.028520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.029215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.030818 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1263/2528] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:15 2022] DELETE /identity/v3/projects/1b76f9eee7a14eb581dcdc19101fc774 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:16.034921 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:16.036336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:16.050347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c95164-51d5-4613-8953-cb13e40938bb admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:16.062973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:16.116771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:16.117408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:16.117679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:16.117965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] PATH_INFO: `/v3/users/947d3e1d6e994aacb03df034a23c42fd` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:16.119098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=947d3e1d6e994aacb03df034a23c42fd)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:16.121498 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:16.175313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947d3e1d6e994aacb03df034a23c42fd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.176102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947d3e1d6e994aacb03df034a23c42fd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.184014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947d3e1d6e994aacb03df034a23c42fd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.196718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947d3e1d6e994aacb03df034a23c42fd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.197628 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947d3e1d6e994aacb03df034a23c42fd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.214090 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1266/2529] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:16 2022] DELETE /identity/v3/users/947d3e1d6e994aacb03df034a23c42fd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:16.692897 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:16.695044 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:16.708779 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79023c9c-5c19-4d38-bb36-3b13b5f65ad7 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:16.782125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:16.782863 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:16.783249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:16.784252 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] PATH_INFO: `/v3/projects/ce06db306562401dafdefc49d5a58b36` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:16.789092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce06db306562401dafdefc49d5a58b36)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:16.791072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:16.863285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce06db306562401dafdefc49d5a58b36', 'request_id': 'req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.872346 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce06db306562401dafdefc49d5a58b36', 'request_id': 'req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.878667 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce06db306562401dafdefc49d5a58b36', 'request_id': 'req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.894126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] The token cache is being invalidate because project ce06db306562401dafdefc49d5a58b36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:16.894407 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.895249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:16.897108 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1264/2530] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:16 2022] DELETE /identity/v3/projects/ce06db306562401dafdefc49d5a58b36 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:21.365282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:21.367012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:21.461604 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dcc29a-cbd1-47af-8321-103b0123a130 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:21.491451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:21.514498 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac16b72-f35c-4be2-83d1-17383f039f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:21.515109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac16b72-f35c-4be2-83d1-17383f039f92 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:21.515511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac16b72-f35c-4be2-83d1-17383f039f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:21.515793 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac16b72-f35c-4be2-83d1-17383f039f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:21.517583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6ac16b72-f35c-4be2-83d1-17383f039f92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:21.580535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac16b72-f35c-4be2-83d1-17383f039f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:21.582012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac16b72-f35c-4be2-83d1-17383f039f92 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:21.591947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1267/2531] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:24.892144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:24.893697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:24.907377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba1cf8-72e6-4b12-a500-457ff8e5f9bf admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:24.912860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d6fc1-04fe-4b4c-a873-b598cfbdb272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:24.913501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d6fc1-04fe-4b4c-a873-b598cfbdb272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:24.913776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d6fc1-04fe-4b4c-a873-b598cfbdb272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:24.914045 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d6fc1-04fe-4b4c-a873-b598cfbdb272 admin admin] PATH_INFO: `/v3/users/4cfac62e1c304167abb971815d5da358` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:24.915272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d6fc1-04fe-4b4c-a873-b598cfbdb272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cfac62e1c304167abb971815d5da358)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:24.917297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d6fc1-04fe-4b4c-a873-b598cfbdb272 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:24.964362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:24.979681 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a65d6fc1-04fe-4b4c-a873-b598cfbdb272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cfac62e1c304167abb971815d5da358'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:24.987326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a65d6fc1-04fe-4b4c-a873-b598cfbdb272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cfac62e1c304167abb971815d5da358'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:25.000785 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a65d6fc1-04fe-4b4c-a873-b598cfbdb272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cfac62e1c304167abb971815d5da358'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:25.011209 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a65d6fc1-04fe-4b4c-a873-b598cfbdb272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cfac62e1c304167abb971815d5da358'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:25.011964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a65d6fc1-04fe-4b4c-a873-b598cfbdb272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cfac62e1c304167abb971815d5da358'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:25.019106 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1265/2532] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:24 2022] DELETE /identity/v3/users/4cfac62e1c304167abb971815d5da358 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:25.203434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac16b72-f35c-4be2-83d1-17383f039f92 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:25.217662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac16b72-f35c-4be2-83d1-17383f039f92 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:25.280029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14441eb-ac36-4213-b6b5-7a99cb404e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:25.280937 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14441eb-ac36-4213-b6b5-7a99cb404e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:25.281230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14441eb-ac36-4213-b6b5-7a99cb404e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:25.281762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14441eb-ac36-4213-b6b5-7a99cb404e78 admin admin] PATH_INFO: `/v3/projects/1abe4f1508864f77b30c4c635c368f02` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:25.283193 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14441eb-ac36-4213-b6b5-7a99cb404e78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1abe4f1508864f77b30c4c635c368f02)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:25.285393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14441eb-ac36-4213-b6b5-7a99cb404e78 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:25.335743 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c14441eb-ac36-4213-b6b5-7a99cb404e78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1abe4f1508864f77b30c4c635c368f02', 'request_id': 'req-c14441eb-ac36-4213-b6b5-7a99cb404e78'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:25.345159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c14441eb-ac36-4213-b6b5-7a99cb404e78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1abe4f1508864f77b30c4c635c368f02', 'request_id': 'req-c14441eb-ac36-4213-b6b5-7a99cb404e78'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:25.352627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c14441eb-ac36-4213-b6b5-7a99cb404e78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1abe4f1508864f77b30c4c635c368f02', 'request_id': 'req-c14441eb-ac36-4213-b6b5-7a99cb404e78'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:25.364486 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c14441eb-ac36-4213-b6b5-7a99cb404e78 admin admin] The token cache is being invalidate because project 1abe4f1508864f77b30c4c635c368f02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:25.364803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c14441eb-ac36-4213-b6b5-7a99cb404e78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:25.365491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c14441eb-ac36-4213-b6b5-7a99cb404e78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:25.367027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1268/2533] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:25 2022] DELETE /identity/v3/projects/1abe4f1508864f77b30c4c635c368f02 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:25.372779 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc9b86-87d1-4cfc-b592-4e26494ea0ed None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:25.373230 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc9b86-87d1-4cfc-b592-4e26494ea0ed None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:25.373603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc9b86-87d1-4cfc-b592-4e26494ea0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:25.434536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-57fc9b86-87d1-4cfc-b592-4e26494ea0ed None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:25.435787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-57fc9b86-87d1-4cfc-b592-4e26494ea0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:25.499206 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1266/2534] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:25.507282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71600a28-f331-4dca-8604-7fccd57b9789 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:25.507567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71600a28-f331-4dca-8604-7fccd57b9789 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:25.507840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71600a28-f331-4dca-8604-7fccd57b9789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:25.528470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-71600a28-f331-4dca-8604-7fccd57b9789 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:25.529745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-71600a28-f331-4dca-8604-7fccd57b9789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:25.540977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1269/2535] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:25.543102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc9b86-87d1-4cfc-b592-4e26494ea0ed None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:25.553267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc9b86-87d1-4cfc-b592-4e26494ea0ed None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:25.558253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623611f6-2a73-4fd4-a3da-9614bf61e9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:25.558934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623611f6-2a73-4fd4-a3da-9614bf61e9bf admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:25.559271 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623611f6-2a73-4fd4-a3da-9614bf61e9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:25.559639 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623611f6-2a73-4fd4-a3da-9614bf61e9bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:25.560365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623611f6-2a73-4fd4-a3da-9614bf61e9bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:25.560781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623611f6-2a73-4fd4-a3da-9614bf61e9bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:25.562439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623611f6-2a73-4fd4-a3da-9614bf61e9bf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:25.574448 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1267/2536] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:26:25.578974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71600a28-f331-4dca-8604-7fccd57b9789 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:25.584968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:25.589109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71600a28-f331-4dca-8604-7fccd57b9789 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:25.596022 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151bce7-a1b6-4952-9efb-dd8c50f2dd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:25.596746 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151bce7-a1b6-4952-9efb-dd8c50f2dd12 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:25.597038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151bce7-a1b6-4952-9efb-dd8c50f2dd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:25.597427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151bce7-a1b6-4952-9efb-dd8c50f2dd12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:25.598306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151bce7-a1b6-4952-9efb-dd8c50f2dd12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:25.601024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151bce7-a1b6-4952-9efb-dd8c50f2dd12 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:25.632259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1270/2537] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:25 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:26:25.636365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623611f6-2a73-4fd4-a3da-9614bf61e9bf admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:25.646685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623611f6-2a73-4fd4-a3da-9614bf61e9bf admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:25.706352 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8e659-1092-46b4-8410-f585ef9d1b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:25.706973 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8e659-1092-46b4-8410-f585ef9d1b1b admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:25.707251 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8e659-1092-46b4-8410-f585ef9d1b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:25.707516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8e659-1092-46b4-8410-f585ef9d1b1b admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:25.708182 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8e659-1092-46b4-8410-f585ef9d1b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:25.710157 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8e659-1092-46b4-8410-f585ef9d1b1b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:25.720905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:25.740566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1268/2538] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:25 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:25.744459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151bce7-a1b6-4952-9efb-dd8c50f2dd12 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:25.755832 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151bce7-a1b6-4952-9efb-dd8c50f2dd12 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:25.760549 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd637595-e5c2-49c4-b1db-9449c1ded0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:25.761335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd637595-e5c2-49c4-b1db-9449c1ded0da admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:25.761643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd637595-e5c2-49c4-b1db-9449c1ded0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:25.761913 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd637595-e5c2-49c4-b1db-9449c1ded0da admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:25.762599 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd637595-e5c2-49c4-b1db-9449c1ded0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:25.764346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd637595-e5c2-49c4-b1db-9449c1ded0da admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:25.774516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1271/2539] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:25.778639 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8e659-1092-46b4-8410-f585ef9d1b1b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:25.789712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8e659-1092-46b4-8410-f585ef9d1b1b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:25.794482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919b4fd-ceec-49c3-b049-4f92fa2c91fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:25.795051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919b4fd-ceec-49c3-b049-4f92fa2c91fe admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:25.795323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919b4fd-ceec-49c3-b049-4f92fa2c91fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:25.795588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919b4fd-ceec-49c3-b049-4f92fa2c91fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:25.796258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919b4fd-ceec-49c3-b049-4f92fa2c91fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:25.797911 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919b4fd-ceec-49c3-b049-4f92fa2c91fe admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:25.808705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1269/2540] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:25.813087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd637595-e5c2-49c4-b1db-9449c1ded0da admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:25.822931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd637595-e5c2-49c4-b1db-9449c1ded0da admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:25.827524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a5c56-24bd-48e2-8fd0-7636ff7c8de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:25.828180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a5c56-24bd-48e2-8fd0-7636ff7c8de2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:25.828454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a5c56-24bd-48e2-8fd0-7636ff7c8de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:25.828768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a5c56-24bd-48e2-8fd0-7636ff7c8de2 admin admin] PATH_INFO: `/v3/projects/7012146d99014fddb3d5a8c2e70a419e/users/6dcfaaef5d344a1295ea2e60ba6a075a/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:25.843276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a5c56-24bd-48e2-8fd0-7636ff7c8de2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7012146d99014fddb3d5a8c2e70a419e, user_id=6dcfaaef5d344a1295ea2e60ba6a075a, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:25.845994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a5c56-24bd-48e2-8fd0-7636ff7c8de2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:25.860395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1272/2541] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:25 2022] PUT /identity/v3/projects/7012146d99014fddb3d5a8c2e70a419e/users/6dcfaaef5d344a1295ea2e60ba6a075a/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:25.874515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1ae35-d387-4f61-85fc-5649881426b9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:25.874763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1ae35-d387-4f61-85fc-5649881426b9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:25.875053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1ae35-d387-4f61-85fc-5649881426b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:25.905632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c1b1ae35-d387-4f61-85fc-5649881426b9 None None] MFA Rules not processed for user `6dcfaaef5d344a1295ea2e60ba6a075a`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:25.906905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c1b1ae35-d387-4f61-85fc-5649881426b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:25.976834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1270/2542] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:25.989164 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a5c56-24bd-48e2-8fd0-7636ff7c8de2 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:25.991204 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7e0a5c56-24bd-48e2-8fd0-7636ff7c8de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:26.070196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a5c56-24bd-48e2-8fd0-7636ff7c8de2 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:26.077159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5fe6b5-8ca1-4fc8-8886-b322578e6d67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:26.077772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fe6b5-8ca1-4fc8-8886-b322578e6d67 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:26.078046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fe6b5-8ca1-4fc8-8886-b322578e6d67 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:26.078306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fe6b5-8ca1-4fc8-8886-b322578e6d67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:26.087049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5fe6b5-8ca1-4fc8-8886-b322578e6d67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:26.088701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5fe6b5-8ca1-4fc8-8886-b322578e6d67 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:26.097988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1273/2543] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:28.894100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1ae35-d387-4f61-85fc-5649881426b9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:28.966968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1ae35-d387-4f61-85fc-5649881426b9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:28.972636 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:28.973264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:28.973532 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:28.973798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] PATH_INFO: `/v3/users/6dcfaaef5d344a1295ea2e60ba6a075a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:28.974982 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dcfaaef5d344a1295ea2e60ba6a075a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:28.976927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.027726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dcfaaef5d344a1295ea2e60ba6a075a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.034745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dcfaaef5d344a1295ea2e60ba6a075a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.047531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dcfaaef5d344a1295ea2e60ba6a075a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.049170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:29.057397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dcfaaef5d344a1295ea2e60ba6a075a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.058225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dcfaaef5d344a1295ea2e60ba6a075a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.065462 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1271/2544] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:28 2022] DELETE /identity/v3/users/6dcfaaef5d344a1295ea2e60ba6a075a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:29.070222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3196-f180-4723-86ed-4ffa6d1027e1 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.070700 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3196-f180-4723-86ed-4ffa6d1027e1 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.070796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd3196-f180-4723-86ed-4ffa6d1027e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.090995 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-23cd3196-f180-4723-86ed-4ffa6d1027e1 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:29.092893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-23cd3196-f180-4723-86ed-4ffa6d1027e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:29.168511 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1274/2545] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:29.179329 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.180959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:29.244247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59736444-39ed-4db1-833f-23855a1d8890 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.250540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17974cce-b404-470c-aaa9-096fa681271e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.251159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17974cce-b404-470c-aaa9-096fa681271e service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.251430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17974cce-b404-470c-aaa9-096fa681271e service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.251709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17974cce-b404-470c-aaa9-096fa681271e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.262899 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17974cce-b404-470c-aaa9-096fa681271e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.264536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17974cce-b404-470c-aaa9-096fa681271e service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.275163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1272/2546] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:29.569676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd3196-f180-4723-86ed-4ffa6d1027e1 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.571309 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-23cd3196-f180-4723-86ed-4ffa6d1027e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:29.583711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd3196-f180-4723-86ed-4ffa6d1027e1 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.588943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68345455-5ea5-46d0-969f-0fcf97ce8c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.589589 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68345455-5ea5-46d0-969f-0fcf97ce8c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.589865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68345455-5ea5-46d0-969f-0fcf97ce8c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.590123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68345455-5ea5-46d0-969f-0fcf97ce8c69 admin admin] PATH_INFO: `/v3/projects/7012146d99014fddb3d5a8c2e70a419e` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.591475 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68345455-5ea5-46d0-969f-0fcf97ce8c69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7012146d99014fddb3d5a8c2e70a419e)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.593735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68345455-5ea5-46d0-969f-0fcf97ce8c69 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.631643 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:29.649900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-68345455-5ea5-46d0-969f-0fcf97ce8c69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7012146d99014fddb3d5a8c2e70a419e', 'request_id': 'req-68345455-5ea5-46d0-969f-0fcf97ce8c69'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.661169 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-68345455-5ea5-46d0-969f-0fcf97ce8c69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7012146d99014fddb3d5a8c2e70a419e', 'request_id': 'req-68345455-5ea5-46d0-969f-0fcf97ce8c69'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.668019 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-68345455-5ea5-46d0-969f-0fcf97ce8c69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7012146d99014fddb3d5a8c2e70a419e', 'request_id': 'req-68345455-5ea5-46d0-969f-0fcf97ce8c69'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.679503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-68345455-5ea5-46d0-969f-0fcf97ce8c69 admin admin] The token cache is being invalidate because project 7012146d99014fddb3d5a8c2e70a419e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:29.679792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-68345455-5ea5-46d0-969f-0fcf97ce8c69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.680455 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-68345455-5ea5-46d0-969f-0fcf97ce8c69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:29.681998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1275/2547] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:29 2022] DELETE /identity/v3/projects/7012146d99014fddb3d5a8c2e70a419e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:29.687111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404035b9-6916-4d00-9cf9-ce78ef61eab6 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.687377 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404035b9-6916-4d00-9cf9-ce78ef61eab6 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.687647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404035b9-6916-4d00-9cf9-ce78ef61eab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.707831 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-404035b9-6916-4d00-9cf9-ce78ef61eab6 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:29.709139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-404035b9-6916-4d00-9cf9-ce78ef61eab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:29.772642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1273/2548] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:29.784281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626b01d-5386-4a09-9326-fc7c806f6b87 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.784565 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626b01d-5386-4a09-9326-fc7c806f6b87 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.784875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626b01d-5386-4a09-9326-fc7c806f6b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.808089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6626b01d-5386-4a09-9326-fc7c806f6b87 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:29.809416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6626b01d-5386-4a09-9326-fc7c806f6b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:29.818433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1276/2549] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:29.823370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404035b9-6916-4d00-9cf9-ce78ef61eab6 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.833987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404035b9-6916-4d00-9cf9-ce78ef61eab6 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.842336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90a2fb-7b7e-4274-a49c-3138c878245c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.842946 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90a2fb-7b7e-4274-a49c-3138c878245c admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.843217 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90a2fb-7b7e-4274-a49c-3138c878245c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.843484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90a2fb-7b7e-4274-a49c-3138c878245c admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.844132 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90a2fb-7b7e-4274-a49c-3138c878245c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:29.844546 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90a2fb-7b7e-4274-a49c-3138c878245c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.846204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90a2fb-7b7e-4274-a49c-3138c878245c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.859022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1274/2550] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 15:26:29.863669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626b01d-5386-4a09-9326-fc7c806f6b87 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.874436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626b01d-5386-4a09-9326-fc7c806f6b87 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:29.880322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b68af-26ce-443d-8b6d-79700d2026bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:29.881049 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b68af-26ce-443d-8b6d-79700d2026bd admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:29.881314 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b68af-26ce-443d-8b6d-79700d2026bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:29.881573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b68af-26ce-443d-8b6d-79700d2026bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:29.882259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b68af-26ce-443d-8b6d-79700d2026bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:29.884102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b68af-26ce-443d-8b6d-79700d2026bd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:29.925387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1277/2551] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:29 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:29.932226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90a2fb-7b7e-4274-a49c-3138c878245c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:29.944442 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90a2fb-7b7e-4274-a49c-3138c878245c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.006447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6aedb-f343-406b-9a8c-0183a5ecdf12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.007074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6aedb-f343-406b-9a8c-0183a5ecdf12 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.007338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6aedb-f343-406b-9a8c-0183a5ecdf12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.007601 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6aedb-f343-406b-9a8c-0183a5ecdf12 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.008231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6aedb-f343-406b-9a8c-0183a5ecdf12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.010143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6aedb-f343-406b-9a8c-0183a5ecdf12 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.033508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1275/2552] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:29 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:30.038048 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b68af-26ce-443d-8b6d-79700d2026bd admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.051374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b68af-26ce-443d-8b6d-79700d2026bd admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.057364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96431866-e5e1-4217-ba6c-4dcaef39b2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.057982 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96431866-e5e1-4217-ba6c-4dcaef39b2ad admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.058255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96431866-e5e1-4217-ba6c-4dcaef39b2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.058520 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96431866-e5e1-4217-ba6c-4dcaef39b2ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.059225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96431866-e5e1-4217-ba6c-4dcaef39b2ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.060933 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96431866-e5e1-4217-ba6c-4dcaef39b2ad admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.071829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1278/2553] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:30.076177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6aedb-f343-406b-9a8c-0183a5ecdf12 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.086694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6aedb-f343-406b-9a8c-0183a5ecdf12 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.091798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5b406-5d51-4ca6-bec3-8fb03ad4e9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.092402 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5b406-5d51-4ca6-bec3-8fb03ad4e9d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.092710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5b406-5d51-4ca6-bec3-8fb03ad4e9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.092981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5b406-5d51-4ca6-bec3-8fb03ad4e9d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.093629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5b406-5d51-4ca6-bec3-8fb03ad4e9d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.095225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5b406-5d51-4ca6-bec3-8fb03ad4e9d3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.100527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:30.110593 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1276/2554] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:30.112183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96431866-e5e1-4217-ba6c-4dcaef39b2ad admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.125468 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96431866-e5e1-4217-ba6c-4dcaef39b2ad admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.130890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34da9be-d165-4f77-933d-91c5c2634dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.131848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34da9be-d165-4f77-933d-91c5c2634dfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.132367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34da9be-d165-4f77-933d-91c5c2634dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.132784 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34da9be-d165-4f77-933d-91c5c2634dfd admin admin] PATH_INFO: `/v3/projects/61e375b8c95541f6b69a17da2214a341/users/2634ff0b6e984822ab001bb33fe70df0/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.149864 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34da9be-d165-4f77-933d-91c5c2634dfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e375b8c95541f6b69a17da2214a341, user_id=2634ff0b6e984822ab001bb33fe70df0, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.152502 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34da9be-d165-4f77-933d-91c5c2634dfd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.167881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1279/2555] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:30 2022] PUT /identity/v3/projects/61e375b8c95541f6b69a17da2214a341/users/2634ff0b6e984822ab001bb33fe70df0/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:30.179286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b0785-49f9-4fa4-8ce1-08206bf7ab5a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.179568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b0785-49f9-4fa4-8ce1-08206bf7ab5a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.179836 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b0785-49f9-4fa4-8ce1-08206bf7ab5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.209712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5e8b0785-49f9-4fa4-8ce1-08206bf7ab5a None None] MFA Rules not processed for user `2634ff0b6e984822ab001bb33fe70df0`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:30.210952 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5e8b0785-49f9-4fa4-8ce1-08206bf7ab5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:30.279170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1277/2556] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:30.284366 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34da9be-d165-4f77-933d-91c5c2634dfd admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.350367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34da9be-d165-4f77-933d-91c5c2634dfd admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.356088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8526e93-f579-40cd-a5eb-653f005299a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.356827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8526e93-f579-40cd-a5eb-653f005299a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.357109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8526e93-f579-40cd-a5eb-653f005299a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.357472 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8526e93-f579-40cd-a5eb-653f005299a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.358044 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8526e93-f579-40cd-a5eb-653f005299a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.359948 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8526e93-f579-40cd-a5eb-653f005299a1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.391422 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1280/2557] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:30 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:30.395703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b0785-49f9-4fa4-8ce1-08206bf7ab5a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.406592 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b0785-49f9-4fa4-8ce1-08206bf7ab5a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.475835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787bd06-1477-4294-ac2d-54850f4fb24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.476460 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787bd06-1477-4294-ac2d-54850f4fb24f admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.476767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787bd06-1477-4294-ac2d-54850f4fb24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.477033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787bd06-1477-4294-ac2d-54850f4fb24f admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.477701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787bd06-1477-4294-ac2d-54850f4fb24f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.479660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787bd06-1477-4294-ac2d-54850f4fb24f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.505395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1278/2558] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:30 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:30.510231 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8526e93-f579-40cd-a5eb-653f005299a1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.520816 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8526e93-f579-40cd-a5eb-653f005299a1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.525698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03305e-1d58-4183-a71f-31227cdadc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.526296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03305e-1d58-4183-a71f-31227cdadc14 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.526573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03305e-1d58-4183-a71f-31227cdadc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.526890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03305e-1d58-4183-a71f-31227cdadc14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.527530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03305e-1d58-4183-a71f-31227cdadc14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.529476 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03305e-1d58-4183-a71f-31227cdadc14 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.540797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1281/2559] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:30.545219 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787bd06-1477-4294-ac2d-54850f4fb24f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.556021 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787bd06-1477-4294-ac2d-54850f4fb24f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.561211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52094716-1e68-4928-a80d-01eed6b9fdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.561842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52094716-1e68-4928-a80d-01eed6b9fdf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.562111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52094716-1e68-4928-a80d-01eed6b9fdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.562389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52094716-1e68-4928-a80d-01eed6b9fdf2 admin admin] PATH_INFO: `/v3/projects/311d13f047414a86a64dad6600823c0e/users/ca6724ce99a44daaa18812c981d3bf64/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.578138 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52094716-1e68-4928-a80d-01eed6b9fdf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=311d13f047414a86a64dad6600823c0e, user_id=ca6724ce99a44daaa18812c981d3bf64, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.580794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52094716-1e68-4928-a80d-01eed6b9fdf2 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.596899 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1279/2560] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:30 2022] PUT /identity/v3/projects/311d13f047414a86a64dad6600823c0e/users/ca6724ce99a44daaa18812c981d3bf64/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:30.601089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03305e-1d58-4183-a71f-31227cdadc14 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.611974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03305e-1d58-4183-a71f-31227cdadc14 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.636903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:30.675173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef93cd2-ef2e-4c69-af08-c00cad2ca06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.675689 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef93cd2-ef2e-4c69-af08-c00cad2ca06c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.675954 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef93cd2-ef2e-4c69-af08-c00cad2ca06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.676218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef93cd2-ef2e-4c69-af08-c00cad2ca06c admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.676894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef93cd2-ef2e-4c69-af08-c00cad2ca06c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.678495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef93cd2-ef2e-4c69-af08-c00cad2ca06c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.688692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1282/2561] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:30.694710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52094716-1e68-4928-a80d-01eed6b9fdf2 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.705700 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52094716-1e68-4928-a80d-01eed6b9fdf2 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.712968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf44026-61e7-4a90-bc11-f21037c68e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.713995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf44026-61e7-4a90-bc11-f21037c68e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.714429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf44026-61e7-4a90-bc11-f21037c68e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.714863 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf44026-61e7-4a90-bc11-f21037c68e82 admin admin] PATH_INFO: `/v3/projects/311d13f047414a86a64dad6600823c0e/users/ca6724ce99a44daaa18812c981d3bf64/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.718051 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf44026-61e7-4a90-bc11-f21037c68e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=311d13f047414a86a64dad6600823c0e, user_id=ca6724ce99a44daaa18812c981d3bf64, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.720754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf44026-61e7-4a90-bc11-f21037c68e82 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.731659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1280/2562] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:30 2022] PUT /identity/v3/projects/311d13f047414a86a64dad6600823c0e/users/ca6724ce99a44daaa18812c981d3bf64/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:26:30.742297 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82a115-3869-4718-ae1d-8e134f7b0487 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.742585 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82a115-3869-4718-ae1d-8e134f7b0487 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.742854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82a115-3869-4718-ae1d-8e134f7b0487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.777005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-8a82a115-3869-4718-ae1d-8e134f7b0487 None None] MFA Rules not processed for user `ca6724ce99a44daaa18812c981d3bf64`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:30.778953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-8a82a115-3869-4718-ae1d-8e134f7b0487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:30.859974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1283/2563] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:30.872158 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf44026-61e7-4a90-bc11-f21037c68e82 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:30.874167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cdf44026-61e7-4a90-bc11-f21037c68e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:30.922740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:30.958478 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf44026-61e7-4a90-bc11-f21037c68e82 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:30.962939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b0a6a-edde-4712-bb9c-fe2e8cfa0ad5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:30.963547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b0a6a-edde-4712-bb9c-fe2e8cfa0ad5 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:30.963824 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b0a6a-edde-4712-bb9c-fe2e8cfa0ad5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:30.964091 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b0a6a-edde-4712-bb9c-fe2e8cfa0ad5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:30.973582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b0a6a-edde-4712-bb9c-fe2e8cfa0ad5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:30.975000 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b0a6a-edde-4712-bb9c-fe2e8cfa0ad5 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:30.984824 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1281/2564] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:31.423047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82a115-3869-4718-ae1d-8e134f7b0487 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:31.498675 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82a115-3869-4718-ae1d-8e134f7b0487 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:31.504056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:31.504739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:31.505014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:31.505308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] PATH_INFO: `/v3/users/2634ff0b6e984822ab001bb33fe70df0` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:31.506709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2634ff0b6e984822ab001bb33fe70df0)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:31.508816 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:31.565053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2634ff0b6e984822ab001bb33fe70df0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:31.565807 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2634ff0b6e984822ab001bb33fe70df0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:31.567881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:31.574075 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2634ff0b6e984822ab001bb33fe70df0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:31.585771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2634ff0b6e984822ab001bb33fe70df0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:31.586392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2634ff0b6e984822ab001bb33fe70df0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:31.600414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1284/2565] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:31 2022] DELETE /identity/v3/users/2634ff0b6e984822ab001bb33fe70df0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:31.605039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7735931-a420-43e8-8d4d-4c7592fba5e1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:31.605311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7735931-a420-43e8-8d4d-4c7592fba5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:31.605573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7735931-a420-43e8-8d4d-4c7592fba5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:31.624694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b7735931-a420-43e8-8d4d-4c7592fba5e1 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:31.625877 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b7735931-a420-43e8-8d4d-4c7592fba5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:31.686876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1282/2566] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:31.697020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:31.698634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:31.757512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f861f9-0b1f-4503-bfba-01371996d4b3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:31.761882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ebbfd-da4f-4d1c-96dc-7cf1d2f9c671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:31.762497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ebbfd-da4f-4d1c-96dc-7cf1d2f9c671 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:31.762769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ebbfd-da4f-4d1c-96dc-7cf1d2f9c671 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:31.763037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ebbfd-da4f-4d1c-96dc-7cf1d2f9c671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:31.773408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ebbfd-da4f-4d1c-96dc-7cf1d2f9c671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:31.774844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ebbfd-da4f-4d1c-96dc-7cf1d2f9c671 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:31.784670 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1285/2567] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:32.095434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7735931-a420-43e8-8d4d-4c7592fba5e1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:32.097028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b7735931-a420-43e8-8d4d-4c7592fba5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:32.111243 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7735931-a420-43e8-8d4d-4c7592fba5e1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:32.116703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:32.117346 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:32.117618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:32.117887 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] PATH_INFO: `/v3/projects/61e375b8c95541f6b69a17da2214a341` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:32.119133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] RBAC: Authorizing `identity:delete_project(project_id=61e375b8c95541f6b69a17da2214a341)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:32.121076 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:32.170619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61e375b8c95541f6b69a17da2214a341', 'request_id': 'req-b3ce6399-54f4-4506-8216-8756d348a1de'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.176856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61e375b8c95541f6b69a17da2214a341', 'request_id': 'req-b3ce6399-54f4-4506-8216-8756d348a1de'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.182330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61e375b8c95541f6b69a17da2214a341', 'request_id': 'req-b3ce6399-54f4-4506-8216-8756d348a1de'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.197913 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] The token cache is being invalidate because project 61e375b8c95541f6b69a17da2214a341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:32.198191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.198814 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.200308 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1283/2568] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:32 2022] DELETE /identity/v3/projects/61e375b8c95541f6b69a17da2214a341 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:32.204614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ebbfd-da4f-4d1c-96dc-7cf1d2f9c671 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:32.214778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ebbfd-da4f-4d1c-96dc-7cf1d2f9c671 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:32.278470 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ec7ed-3741-4afc-ae23-8fdcd4acb7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:32.279096 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ec7ed-3741-4afc-ae23-8fdcd4acb7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:32.279437 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ec7ed-3741-4afc-ae23-8fdcd4acb7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:32.279708 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ec7ed-3741-4afc-ae23-8fdcd4acb7dd admin admin] PATH_INFO: `/v3/users/ca6724ce99a44daaa18812c981d3bf64` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:32.281002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ec7ed-3741-4afc-ae23-8fdcd4acb7dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca6724ce99a44daaa18812c981d3bf64)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:32.282992 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ec7ed-3741-4afc-ae23-8fdcd4acb7dd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:32.332221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5b7ec7ed-3741-4afc-ae23-8fdcd4acb7dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca6724ce99a44daaa18812c981d3bf64'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.332892 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5b7ec7ed-3741-4afc-ae23-8fdcd4acb7dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca6724ce99a44daaa18812c981d3bf64'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.339581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5b7ec7ed-3741-4afc-ae23-8fdcd4acb7dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca6724ce99a44daaa18812c981d3bf64'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.350381 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:32.351663 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5b7ec7ed-3741-4afc-ae23-8fdcd4acb7dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca6724ce99a44daaa18812c981d3bf64'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.352412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5b7ec7ed-3741-4afc-ae23-8fdcd4acb7dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca6724ce99a44daaa18812c981d3bf64'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.371794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1286/2569] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:32 2022] DELETE /identity/v3/users/ca6724ce99a44daaa18812c981d3bf64 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:32.649733 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:32.651334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:32.666322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce6399-54f4-4506-8216-8756d348a1de admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:32.713762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:32.727361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f0c59-7f80-45de-8b7b-f72c7925efef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:32.727990 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f0c59-7f80-45de-8b7b-f72c7925efef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:32.728258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f0c59-7f80-45de-8b7b-f72c7925efef admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:32.728520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f0c59-7f80-45de-8b7b-f72c7925efef admin admin] PATH_INFO: `/v3/projects/311d13f047414a86a64dad6600823c0e` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:32.729662 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f0c59-7f80-45de-8b7b-f72c7925efef admin admin] RBAC: Authorizing `identity:delete_project(project_id=311d13f047414a86a64dad6600823c0e)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:32.731602 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f0c59-7f80-45de-8b7b-f72c7925efef admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:32.783942 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-948f0c59-7f80-45de-8b7b-f72c7925efef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '311d13f047414a86a64dad6600823c0e', 'request_id': 'req-948f0c59-7f80-45de-8b7b-f72c7925efef'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.791472 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-948f0c59-7f80-45de-8b7b-f72c7925efef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '311d13f047414a86a64dad6600823c0e', 'request_id': 'req-948f0c59-7f80-45de-8b7b-f72c7925efef'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.799290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-948f0c59-7f80-45de-8b7b-f72c7925efef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '311d13f047414a86a64dad6600823c0e', 'request_id': 'req-948f0c59-7f80-45de-8b7b-f72c7925efef'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.816563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-948f0c59-7f80-45de-8b7b-f72c7925efef admin admin] The token cache is being invalidate because project 311d13f047414a86a64dad6600823c0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:32.816880 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-948f0c59-7f80-45de-8b7b-f72c7925efef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.817608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-948f0c59-7f80-45de-8b7b-f72c7925efef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:32.819169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1284/2570] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:32 2022] DELETE /identity/v3/projects/311d13f047414a86a64dad6600823c0e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:32.824835 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704da025-5505-40f0-8efa-7c4acc2dcee9 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:32.825319 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704da025-5505-40f0-8efa-7c4acc2dcee9 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:32.825414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704da025-5505-40f0-8efa-7c4acc2dcee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:32.845742 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-704da025-5505-40f0-8efa-7c4acc2dcee9 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:32.846953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-704da025-5505-40f0-8efa-7c4acc2dcee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:32.911161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1287/2571] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:26:32.920225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64079ab-46ae-4dfd-917d-c345c1947710 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:32.920490 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64079ab-46ae-4dfd-917d-c345c1947710 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:32.920767 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64079ab-46ae-4dfd-917d-c345c1947710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:32.941986 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e64079ab-46ae-4dfd-917d-c345c1947710 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:32.943189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e64079ab-46ae-4dfd-917d-c345c1947710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:32.951750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1285/2572] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:32.956174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704da025-5505-40f0-8efa-7c4acc2dcee9 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:32.966269 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704da025-5505-40f0-8efa-7c4acc2dcee9 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:32.972492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef5787-7bef-4bd8-801c-cf00ed1f3460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:32.973132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef5787-7bef-4bd8-801c-cf00ed1f3460 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:32.973404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef5787-7bef-4bd8-801c-cf00ed1f3460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:32.973669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef5787-7bef-4bd8-801c-cf00ed1f3460 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:32.974315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef5787-7bef-4bd8-801c-cf00ed1f3460 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:32.974612 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef5787-7bef-4bd8-801c-cf00ed1f3460 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:32.976212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef5787-7bef-4bd8-801c-cf00ed1f3460 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:32.988161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1288/2573] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:26:32.992303 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64079ab-46ae-4dfd-917d-c345c1947710 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:33.001957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64079ab-46ae-4dfd-917d-c345c1947710 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:33.006761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fa756-64a5-4a30-83d7-0fedee80c0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:33.007354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fa756-64a5-4a30-83d7-0fedee80c0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:33.007627 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fa756-64a5-4a30-83d7-0fedee80c0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:33.007894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fa756-64a5-4a30-83d7-0fedee80c0b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:33.008522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fa756-64a5-4a30-83d7-0fedee80c0b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:33.010419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fa756-64a5-4a30-83d7-0fedee80c0b5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:33.041123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1286/2574] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:32 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:33.047188 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef5787-7bef-4bd8-801c-cf00ed1f3460 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:33.062313 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef5787-7bef-4bd8-801c-cf00ed1f3460 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:33.128328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d2f50-e00d-4418-89cc-c1e0d3ad561c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:33.128979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d2f50-e00d-4418-89cc-c1e0d3ad561c admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:33.129252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d2f50-e00d-4418-89cc-c1e0d3ad561c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:33.129514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d2f50-e00d-4418-89cc-c1e0d3ad561c admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:33.130150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d2f50-e00d-4418-89cc-c1e0d3ad561c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:33.132076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d2f50-e00d-4418-89cc-c1e0d3ad561c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:33.159036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1289/2575] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:33 2022] POST /identity/v3/users => generated 282 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:26:33.163268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fa756-64a5-4a30-83d7-0fedee80c0b5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:33.174884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fa756-64a5-4a30-83d7-0fedee80c0b5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:33.180520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660007d4-a6c0-43da-abb2-f3891bcb806a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:33.181148 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660007d4-a6c0-43da-abb2-f3891bcb806a admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:33.181429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660007d4-a6c0-43da-abb2-f3891bcb806a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:33.181702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660007d4-a6c0-43da-abb2-f3891bcb806a admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:33.182350 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660007d4-a6c0-43da-abb2-f3891bcb806a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:33.183933 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660007d4-a6c0-43da-abb2-f3891bcb806a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:33.194810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1287/2576] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:33.198822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d2f50-e00d-4418-89cc-c1e0d3ad561c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:33.209709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d2f50-e00d-4418-89cc-c1e0d3ad561c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:33.214492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0495ce6-3784-440f-a060-a35fce4214c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:33.215281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0495ce6-3784-440f-a060-a35fce4214c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:33.215561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0495ce6-3784-440f-a060-a35fce4214c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:33.215829 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0495ce6-3784-440f-a060-a35fce4214c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:33.216580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0495ce6-3784-440f-a060-a35fce4214c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:33.218214 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0495ce6-3784-440f-a060-a35fce4214c5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:33.230259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1290/2577] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:33.235079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660007d4-a6c0-43da-abb2-f3891bcb806a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:33.249825 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660007d4-a6c0-43da-abb2-f3891bcb806a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:33.255690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b61615-95c3-4614-a32c-4339d5251ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:33.256348 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b61615-95c3-4614-a32c-4339d5251ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:33.256631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b61615-95c3-4614-a32c-4339d5251ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:33.256906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b61615-95c3-4614-a32c-4339d5251ce7 admin admin] PATH_INFO: `/v3/projects/e94c309e2d984eed85024748205393c0/users/786957ab3576404a91c8e0cbbd4d8f42/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:33.273955 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b61615-95c3-4614-a32c-4339d5251ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e94c309e2d984eed85024748205393c0, user_id=786957ab3576404a91c8e0cbbd4d8f42, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:33.276613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b61615-95c3-4614-a32c-4339d5251ce7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:33.292869 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1288/2578] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:33 2022] PUT /identity/v3/projects/e94c309e2d984eed85024748205393c0/users/786957ab3576404a91c8e0cbbd4d8f42/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:33.304260 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f565170-f0e6-4bdd-af00-e7e926c7fece None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:33.304652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f565170-f0e6-4bdd-af00-e7e926c7fece None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:33.305043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f565170-f0e6-4bdd-af00-e7e926c7fece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:33.335284 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-4f565170-f0e6-4bdd-af00-e7e926c7fece None None] MFA Rules not processed for user `786957ab3576404a91c8e0cbbd4d8f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:33.336406 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4f565170-f0e6-4bdd-af00-e7e926c7fece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:33.380961 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:33.405034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1291/2579] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:33.409660 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b61615-95c3-4614-a32c-4339d5251ce7 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:33.473617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b61615-95c3-4614-a32c-4339d5251ce7 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:33.479053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a793d69-6919-4aac-96a9-1d3fe396ec5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:33.479662 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a793d69-6919-4aac-96a9-1d3fe396ec5e admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:33.479934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a793d69-6919-4aac-96a9-1d3fe396ec5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:33.480198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a793d69-6919-4aac-96a9-1d3fe396ec5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:33.480874 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a793d69-6919-4aac-96a9-1d3fe396ec5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:33.483337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a793d69-6919-4aac-96a9-1d3fe396ec5e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:33.507428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:33.514749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1289/2580] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:33 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:33.519151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f565170-f0e6-4bdd-af00-e7e926c7fece None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:33.530534 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f565170-f0e6-4bdd-af00-e7e926c7fece None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:33.589510 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1514d7-f2ef-4802-8d11-de79969fb8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:33.590039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1514d7-f2ef-4802-8d11-de79969fb8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:33.590312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1514d7-f2ef-4802-8d11-de79969fb8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:33.590580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1514d7-f2ef-4802-8d11-de79969fb8e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:33.591237 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1514d7-f2ef-4802-8d11-de79969fb8e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:33.593750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1514d7-f2ef-4802-8d11-de79969fb8e1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:33.617817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1292/2581] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:33 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:33.622447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a793d69-6919-4aac-96a9-1d3fe396ec5e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:33.632768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a793d69-6919-4aac-96a9-1d3fe396ec5e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:33.637709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a58a47-60c2-4471-a837-8a6509ed2a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:33.638357 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a58a47-60c2-4471-a837-8a6509ed2a42 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:33.638647 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a58a47-60c2-4471-a837-8a6509ed2a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:33.638905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a58a47-60c2-4471-a837-8a6509ed2a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:33.639572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a58a47-60c2-4471-a837-8a6509ed2a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:33.641169 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a58a47-60c2-4471-a837-8a6509ed2a42 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:33.651922 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1290/2582] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:33.656232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1514d7-f2ef-4802-8d11-de79969fb8e1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:33.666452 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1514d7-f2ef-4802-8d11-de79969fb8e1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:33.673215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f488f6-01db-468e-9627-4ecba6f01b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:33.674129 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f488f6-01db-468e-9627-4ecba6f01b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:33.674485 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f488f6-01db-468e-9627-4ecba6f01b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:33.674864 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f488f6-01db-468e-9627-4ecba6f01b83 admin admin] PATH_INFO: `/v3/projects/9f724180b4a04fc8a8c2d4e1afa375e8/users/8e56f7263ae547a19b6eedf3dbb338c9/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:33.698372 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f488f6-01db-468e-9627-4ecba6f01b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f724180b4a04fc8a8c2d4e1afa375e8, user_id=8e56f7263ae547a19b6eedf3dbb338c9, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:33.702438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f488f6-01db-468e-9627-4ecba6f01b83 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:33.717981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1293/2583] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:33 2022] PUT /identity/v3/projects/9f724180b4a04fc8a8c2d4e1afa375e8/users/8e56f7263ae547a19b6eedf3dbb338c9/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:33.722126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a58a47-60c2-4471-a837-8a6509ed2a42 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:33.732499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a58a47-60c2-4471-a837-8a6509ed2a42 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:33.793195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa37485-4fca-4c04-87ba-e18b6552ea3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:33.793783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa37485-4fca-4c04-87ba-e18b6552ea3f admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:33.794029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa37485-4fca-4c04-87ba-e18b6552ea3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:33.794294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa37485-4fca-4c04-87ba-e18b6552ea3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:33.794987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa37485-4fca-4c04-87ba-e18b6552ea3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:33.796809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa37485-4fca-4c04-87ba-e18b6552ea3f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:33.806925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1291/2584] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:33.811825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f488f6-01db-468e-9627-4ecba6f01b83 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:33.822772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f488f6-01db-468e-9627-4ecba6f01b83 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:33.829265 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd78f66-7fd2-4c2b-9fd4-150cc7b6e7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:33.829922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd78f66-7fd2-4c2b-9fd4-150cc7b6e7ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:33.830198 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd78f66-7fd2-4c2b-9fd4-150cc7b6e7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:33.830465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd78f66-7fd2-4c2b-9fd4-150cc7b6e7ad admin admin] PATH_INFO: `/v3/projects/9f724180b4a04fc8a8c2d4e1afa375e8/users/8e56f7263ae547a19b6eedf3dbb338c9/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:33.832983 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd78f66-7fd2-4c2b-9fd4-150cc7b6e7ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f724180b4a04fc8a8c2d4e1afa375e8, user_id=8e56f7263ae547a19b6eedf3dbb338c9, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:33.835632 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd78f66-7fd2-4c2b-9fd4-150cc7b6e7ad admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:33.847291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1294/2585] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:33 2022] PUT /identity/v3/projects/9f724180b4a04fc8a8c2d4e1afa375e8/users/8e56f7263ae547a19b6eedf3dbb338c9/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:33.858619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97afe2c9-4ade-4ea7-9624-5f5bb5037edf None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:33.858980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97afe2c9-4ade-4ea7-9624-5f5bb5037edf None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:33.859250 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97afe2c9-4ade-4ea7-9624-5f5bb5037edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:33.889143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-97afe2c9-4ade-4ea7-9624-5f5bb5037edf None None] MFA Rules not processed for user `8e56f7263ae547a19b6eedf3dbb338c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:33.890347 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-97afe2c9-4ade-4ea7-9624-5f5bb5037edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:33.960204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1292/2586] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:33.972443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd78f66-7fd2-4c2b-9fd4-150cc7b6e7ad admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:33.974532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0bd78f66-7fd2-4c2b-9fd4-150cc7b6e7ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:34.036184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd78f66-7fd2-4c2b-9fd4-150cc7b6e7ad admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:34.040319 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52a001-0d1a-4f4e-97e1-9881a0d31466 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:34.040944 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a001-0d1a-4f4e-97e1-9881a0d31466 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:34.041216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a001-0d1a-4f4e-97e1-9881a0d31466 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:34.041483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a001-0d1a-4f4e-97e1-9881a0d31466 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:34.051396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52a001-0d1a-4f4e-97e1-9881a0d31466 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:34.052851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52a001-0d1a-4f4e-97e1-9881a0d31466 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:34.062976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1295/2587] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:34.163459 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97afe2c9-4ade-4ea7-9624-5f5bb5037edf None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:34.174864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97afe2c9-4ade-4ea7-9624-5f5bb5037edf None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:34.179649 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ba2fd-ad25-422e-98a8-821afce39e29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:34.180266 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ba2fd-ad25-422e-98a8-821afce39e29 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:34.180545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ba2fd-ad25-422e-98a8-821afce39e29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:34.180853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ba2fd-ad25-422e-98a8-821afce39e29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:34.240610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ba2fd-ad25-422e-98a8-821afce39e29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:34.242072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ba2fd-ad25-422e-98a8-821afce39e29 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:34.253131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1293/2588] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:37.037024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52a001-0d1a-4f4e-97e1-9881a0d31466 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:37.078712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:37.104502 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52a001-0d1a-4f4e-97e1-9881a0d31466 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:37.109525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:37.110143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:37.110413 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:37.110674 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] PATH_INFO: `/v3/users/786957ab3576404a91c8e0cbbd4d8f42` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:37.111808 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] RBAC: Authorizing `identity:delete_user(user_id=786957ab3576404a91c8e0cbbd4d8f42)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:37.113747 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:37.163914 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '786957ab3576404a91c8e0cbbd4d8f42'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.164555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '786957ab3576404a91c8e0cbbd4d8f42'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.171360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '786957ab3576404a91c8e0cbbd4d8f42'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.183964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '786957ab3576404a91c8e0cbbd4d8f42'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.184641 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '786957ab3576404a91c8e0cbbd4d8f42'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.204383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1296/2589] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:37 2022] DELETE /identity/v3/users/786957ab3576404a91c8e0cbbd4d8f42 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:37.209047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccffa2-2524-4d82-ad8f-bc8516224f35 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:37.209519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccffa2-2524-4d82-ad8f-bc8516224f35 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:37.209618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccffa2-2524-4d82-ad8f-bc8516224f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:37.229267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-16ccffa2-2524-4d82-ad8f-bc8516224f35 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:37.230461 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-16ccffa2-2524-4d82-ad8f-bc8516224f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:37.234664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:37.292521 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1294/2590] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:37.302973 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:37.304490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:37.367496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf9f438-ab84-478e-948c-5c809a65313b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:37.374248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7963a-778e-4914-bd5f-fe61d963859c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:37.375105 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7963a-778e-4914-bd5f-fe61d963859c service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:37.375386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7963a-778e-4914-bd5f-fe61d963859c service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:37.375656 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7963a-778e-4914-bd5f-fe61d963859c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:37.390423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7963a-778e-4914-bd5f-fe61d963859c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:37.391855 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7963a-778e-4914-bd5f-fe61d963859c service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:37.401676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1297/2591] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:37.684674 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccffa2-2524-4d82-ad8f-bc8516224f35 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:37.686283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-16ccffa2-2524-4d82-ad8f-bc8516224f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:37.701312 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ccffa2-2524-4d82-ad8f-bc8516224f35 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:37.706474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:37.707372 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:37.707646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:37.707933 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] PATH_INFO: `/v3/projects/e94c309e2d984eed85024748205393c0` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:37.709115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e94c309e2d984eed85024748205393c0)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:37.711155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:37.762074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e94c309e2d984eed85024748205393c0', 'request_id': 'req-d75ea8bf-321e-44ae-a02b-5eafce41b196'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.768369 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e94c309e2d984eed85024748205393c0', 'request_id': 'req-d75ea8bf-321e-44ae-a02b-5eafce41b196'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.773998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e94c309e2d984eed85024748205393c0', 'request_id': 'req-d75ea8bf-321e-44ae-a02b-5eafce41b196'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.792159 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] The token cache is being invalidate because project e94c309e2d984eed85024748205393c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:37.792553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.793561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.796606 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1295/2592] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:37 2022] DELETE /identity/v3/projects/e94c309e2d984eed85024748205393c0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:37.802192 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7963a-778e-4914-bd5f-fe61d963859c service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:37.812698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7963a-778e-4914-bd5f-fe61d963859c service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:37.870075 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25812ac8-c7e1-4d73-901d-df53b45510ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:37.870702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25812ac8-c7e1-4d73-901d-df53b45510ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:37.870978 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25812ac8-c7e1-4d73-901d-df53b45510ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:37.871568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25812ac8-c7e1-4d73-901d-df53b45510ba admin admin] PATH_INFO: `/v3/users/8e56f7263ae547a19b6eedf3dbb338c9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:37.872748 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25812ac8-c7e1-4d73-901d-df53b45510ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e56f7263ae547a19b6eedf3dbb338c9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:37.874769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25812ac8-c7e1-4d73-901d-df53b45510ba admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:37.877115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:37.925397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-25812ac8-c7e1-4d73-901d-df53b45510ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e56f7263ae547a19b6eedf3dbb338c9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.926158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-25812ac8-c7e1-4d73-901d-df53b45510ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e56f7263ae547a19b6eedf3dbb338c9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.933014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-25812ac8-c7e1-4d73-901d-df53b45510ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e56f7263ae547a19b6eedf3dbb338c9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.945773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-25812ac8-c7e1-4d73-901d-df53b45510ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e56f7263ae547a19b6eedf3dbb338c9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.946497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-25812ac8-c7e1-4d73-901d-df53b45510ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e56f7263ae547a19b6eedf3dbb338c9'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:37.961321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1298/2593] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:37 2022] DELETE /identity/v3/users/8e56f7263ae547a19b6eedf3dbb338c9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:38.257609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:38.259180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:38.277194 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ea8bf-321e-44ae-a02b-5eafce41b196 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:38.338729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:38.339386 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:38.339657 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:38.339915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1 admin admin] PATH_INFO: `/v3/projects/9f724180b4a04fc8a8c2d4e1afa375e8` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:38.341353 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f724180b4a04fc8a8c2d4e1afa375e8)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:38.343709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:38.382725 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:38.394436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f724180b4a04fc8a8c2d4e1afa375e8', 'request_id': 'req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:38.401101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f724180b4a04fc8a8c2d4e1afa375e8', 'request_id': 'req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:38.407074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f724180b4a04fc8a8c2d4e1afa375e8', 'request_id': 'req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:38.421280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1 admin admin] The token cache is being invalidate because project 9f724180b4a04fc8a8c2d4e1afa375e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:38.421553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:38.422201 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5a6d2340-17b5-4864-89c7-d5de0bdaf6b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:38.423711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1296/2594] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:38 2022] DELETE /identity/v3/projects/9f724180b4a04fc8a8c2d4e1afa375e8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:38.428919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b80754-c1f0-4b09-8458-dfcd14f3ece6 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:38.429402 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b80754-c1f0-4b09-8458-dfcd14f3ece6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:38.429512 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b80754-c1f0-4b09-8458-dfcd14f3ece6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:38.452425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-98b80754-c1f0-4b09-8458-dfcd14f3ece6 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:38.453563 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-98b80754-c1f0-4b09-8458-dfcd14f3ece6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:38.520897 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1299/2595] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:38.529060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65337397-d5ec-4de0-84a2-2282f89c3a1d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:38.529336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65337397-d5ec-4de0-84a2-2282f89c3a1d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:38.529666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65337397-d5ec-4de0-84a2-2282f89c3a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:38.552029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-65337397-d5ec-4de0-84a2-2282f89c3a1d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:38.553278 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-65337397-d5ec-4de0-84a2-2282f89c3a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:38.567284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1297/2596] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:38.572109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b80754-c1f0-4b09-8458-dfcd14f3ece6 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:38.582283 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b80754-c1f0-4b09-8458-dfcd14f3ece6 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:38.587312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c07cff-3687-4ff2-b1cd-44082dc7cd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:38.587903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c07cff-3687-4ff2-b1cd-44082dc7cd4c admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:38.588179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c07cff-3687-4ff2-b1cd-44082dc7cd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:38.588447 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c07cff-3687-4ff2-b1cd-44082dc7cd4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:38.589157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c07cff-3687-4ff2-b1cd-44082dc7cd4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:38.589458 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c07cff-3687-4ff2-b1cd-44082dc7cd4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:38.591060 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c07cff-3687-4ff2-b1cd-44082dc7cd4c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:38.602575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1300/2597] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:26:38.606916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65337397-d5ec-4de0-84a2-2282f89c3a1d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:38.616365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65337397-d5ec-4de0-84a2-2282f89c3a1d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:38.622027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f653ec0-3fb5-426d-91f5-abc2189ec4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:38.622625 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f653ec0-3fb5-426d-91f5-abc2189ec4de admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:38.622897 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f653ec0-3fb5-426d-91f5-abc2189ec4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:38.623162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f653ec0-3fb5-426d-91f5-abc2189ec4de admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:38.623991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f653ec0-3fb5-426d-91f5-abc2189ec4de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:38.625980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f653ec0-3fb5-426d-91f5-abc2189ec4de admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:38.660640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1298/2598] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:38 2022] POST /identity/v3/projects => generated 361 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:38.666623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c07cff-3687-4ff2-b1cd-44082dc7cd4c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:38.678500 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c07cff-3687-4ff2-b1cd-44082dc7cd4c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:38.739321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce9192-f825-4b6c-9b8d-51f8f71c2be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:38.739843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce9192-f825-4b6c-9b8d-51f8f71c2be5 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:38.740118 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce9192-f825-4b6c-9b8d-51f8f71c2be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:38.740390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce9192-f825-4b6c-9b8d-51f8f71c2be5 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:38.741051 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce9192-f825-4b6c-9b8d-51f8f71c2be5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:38.742967 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce9192-f825-4b6c-9b8d-51f8f71c2be5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:38.761424 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:38.770657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1301/2599] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:38 2022] POST /identity/v3/users => generated 281 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:38.772241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f653ec0-3fb5-426d-91f5-abc2189ec4de admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:38.783340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f653ec0-3fb5-426d-91f5-abc2189ec4de admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:38.788509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc587802-365a-4625-a8f5-a1c11f46bbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:38.789136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc587802-365a-4625-a8f5-a1c11f46bbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:38.789411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc587802-365a-4625-a8f5-a1c11f46bbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:38.789678 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc587802-365a-4625-a8f5-a1c11f46bbd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:38.790320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc587802-365a-4625-a8f5-a1c11f46bbd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:38.791902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc587802-365a-4625-a8f5-a1c11f46bbd0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:38.803603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1299/2600] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:26:38.807567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce9192-f825-4b6c-9b8d-51f8f71c2be5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:38.819072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce9192-f825-4b6c-9b8d-51f8f71c2be5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:38.824734 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e219ce-8428-4998-a150-fc571fbd3a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:38.825331 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e219ce-8428-4998-a150-fc571fbd3a2d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:38.825613 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e219ce-8428-4998-a150-fc571fbd3a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:38.825865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e219ce-8428-4998-a150-fc571fbd3a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:38.826611 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e219ce-8428-4998-a150-fc571fbd3a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:38.828229 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e219ce-8428-4998-a150-fc571fbd3a2d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:38.839168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1302/2601] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:38.843577 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc587802-365a-4625-a8f5-a1c11f46bbd0 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:38.853853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc587802-365a-4625-a8f5-a1c11f46bbd0 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:38.858829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbdb38-04d2-4591-8b92-55db2f08a19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:38.859469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbdb38-04d2-4591-8b92-55db2f08a19a admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:38.859743 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbdb38-04d2-4591-8b92-55db2f08a19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:38.860012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbdb38-04d2-4591-8b92-55db2f08a19a admin admin] PATH_INFO: `/v3/projects/3a4b548575ce46f09159965d007cf3fe/users/f69b6cf0b9024a3194ca706ef4b8aa41/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:38.875577 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbdb38-04d2-4591-8b92-55db2f08a19a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4b548575ce46f09159965d007cf3fe, user_id=f69b6cf0b9024a3194ca706ef4b8aa41, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:38.878483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbdb38-04d2-4591-8b92-55db2f08a19a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:38.894692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1300/2602] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:38 2022] PUT /identity/v3/projects/3a4b548575ce46f09159965d007cf3fe/users/f69b6cf0b9024a3194ca706ef4b8aa41/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:38.905445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5222c7-b6c5-40b1-abbf-58ec2b24a483 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:38.905723 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5222c7-b6c5-40b1-abbf-58ec2b24a483 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:38.905987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5222c7-b6c5-40b1-abbf-58ec2b24a483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:38.934755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ab5222c7-b6c5-40b1-abbf-58ec2b24a483 None None] MFA Rules not processed for user `f69b6cf0b9024a3194ca706ef4b8aa41`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:38.935953 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ab5222c7-b6c5-40b1-abbf-58ec2b24a483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:39.001772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1303/2603] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:26:39.035613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbdb38-04d2-4591-8b92-55db2f08a19a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:39.037741 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fcfbdb38-04d2-4591-8b92-55db2f08a19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:39.107998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbdb38-04d2-4591-8b92-55db2f08a19a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:39.112913 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fae0e-0176-4ce6-83b0-8369bc041d42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:39.113626 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fae0e-0176-4ce6-83b0-8369bc041d42 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:39.113963 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fae0e-0176-4ce6-83b0-8369bc041d42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:39.114234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fae0e-0176-4ce6-83b0-8369bc041d42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:39.123938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2fae0e-0176-4ce6-83b0-8369bc041d42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:39.125411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2fae0e-0176-4ce6-83b0-8369bc041d42 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:39.134125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1301/2604] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:39.149817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5222c7-b6c5-40b1-abbf-58ec2b24a483 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:39.224461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5222c7-b6c5-40b1-abbf-58ec2b24a483 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:39.229397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:39.230024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:39.230296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:39.230557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] PATH_INFO: `/v3/users/f69b6cf0b9024a3194ca706ef4b8aa41` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:39.231692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69b6cf0b9024a3194ca706ef4b8aa41)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:39.233792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:39.293079 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69b6cf0b9024a3194ca706ef4b8aa41'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:39.293669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69b6cf0b9024a3194ca706ef4b8aa41'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:39.301146 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69b6cf0b9024a3194ca706ef4b8aa41'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:39.313034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69b6cf0b9024a3194ca706ef4b8aa41'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:39.313718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69b6cf0b9024a3194ca706ef4b8aa41'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:39.317339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:39.329215 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1304/2605] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:39 2022] DELETE /identity/v3/users/f69b6cf0b9024a3194ca706ef4b8aa41 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:39.333955 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f9254-c31c-4698-bbd6-718ac1a700f3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:39.334480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f9254-c31c-4698-bbd6-718ac1a700f3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:39.334480 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f9254-c31c-4698-bbd6-718ac1a700f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:39.354518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a54f9254-c31c-4698-bbd6-718ac1a700f3 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:39.355709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a54f9254-c31c-4698-bbd6-718ac1a700f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:39.419921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1302/2606] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:39.430575 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:39.432709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:39.505307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c66e5b-ae92-40d3-ac40-f6bae10a2720 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:39.511576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529896db-cb43-4fd4-9113-a009729b3a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:39.512207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529896db-cb43-4fd4-9113-a009729b3a5d service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:39.512479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529896db-cb43-4fd4-9113-a009729b3a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:39.512772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529896db-cb43-4fd4-9113-a009729b3a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:39.523006 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529896db-cb43-4fd4-9113-a009729b3a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:39.524464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529896db-cb43-4fd4-9113-a009729b3a5d service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:39.534826 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1305/2607] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:39.812253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f9254-c31c-4698-bbd6-718ac1a700f3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:39.814012 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a54f9254-c31c-4698-bbd6-718ac1a700f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:39.821209 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:39.827218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f9254-c31c-4698-bbd6-718ac1a700f3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:39.833371 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:39.834172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:39.834443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:39.834712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27 admin admin] PATH_INFO: `/v3/projects/3a4b548575ce46f09159965d007cf3fe` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:39.836151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4b548575ce46f09159965d007cf3fe)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:39.838862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:39.891588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4b548575ce46f09159965d007cf3fe', 'request_id': 'req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:39.897891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4b548575ce46f09159965d007cf3fe', 'request_id': 'req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:39.904260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4b548575ce46f09159965d007cf3fe', 'request_id': 'req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:39.919253 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27 admin admin] The token cache is being invalidate because project 3a4b548575ce46f09159965d007cf3fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:39.919538 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:39.920164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b9abf2fb-fa9a-4430-82cd-cd4bde6c9a27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:39.921755 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1303/2608] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:39 2022] DELETE /identity/v3/projects/3a4b548575ce46f09159965d007cf3fe => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:26:39.927537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e58fa0-1852-400d-8fda-4d3ab81b2bf2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:39.927871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e58fa0-1852-400d-8fda-4d3ab81b2bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:39.928309 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e58fa0-1852-400d-8fda-4d3ab81b2bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:39.952111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d9e58fa0-1852-400d-8fda-4d3ab81b2bf2 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:39.953598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d9e58fa0-1852-400d-8fda-4d3ab81b2bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:40.024195 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1306/2609] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:40.034156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1daed1-c40b-4af0-9a95-9bf503b5d0c4 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.034388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1daed1-c40b-4af0-9a95-9bf503b5d0c4 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.034709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1daed1-c40b-4af0-9a95-9bf503b5d0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.056746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5a1daed1-c40b-4af0-9a95-9bf503b5d0c4 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:40.057873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5a1daed1-c40b-4af0-9a95-9bf503b5d0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:40.066641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1304/2610] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:40.071003 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e58fa0-1852-400d-8fda-4d3ab81b2bf2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:40.081065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e58fa0-1852-400d-8fda-4d3ab81b2bf2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:40.086900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46b46f-e600-457a-8680-0144e06531e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:40.087497 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46b46f-e600-457a-8680-0144e06531e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.087748 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46b46f-e600-457a-8680-0144e06531e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.088003 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46b46f-e600-457a-8680-0144e06531e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.088673 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46b46f-e600-457a-8680-0144e06531e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:40.088987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46b46f-e600-457a-8680-0144e06531e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:40.090572 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf46b46f-e600-457a-8680-0144e06531e4 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:40.110152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1307/2611] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:26:40.113353 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1daed1-c40b-4af0-9a95-9bf503b5d0c4 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:40.127670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1daed1-c40b-4af0-9a95-9bf503b5d0c4 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:40.133530 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee86dd-94d0-402c-b544-cb965ce1a1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:40.134131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee86dd-94d0-402c-b544-cb965ce1a1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.134400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee86dd-94d0-402c-b544-cb965ce1a1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.134659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee86dd-94d0-402c-b544-cb965ce1a1d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.135291 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee86dd-94d0-402c-b544-cb965ce1a1d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:40.137218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee86dd-94d0-402c-b544-cb965ce1a1d3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:40.173042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1305/2612] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:40 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:40.177461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46b46f-e600-457a-8680-0144e06531e4 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:40.187932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf46b46f-e600-457a-8680-0144e06531e4 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:40.249317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2aa4a-2068-4861-8342-6b11745bc9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:40.249921 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2aa4a-2068-4861-8342-6b11745bc9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.250185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2aa4a-2068-4861-8342-6b11745bc9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.250453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2aa4a-2068-4861-8342-6b11745bc9c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.251079 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2aa4a-2068-4861-8342-6b11745bc9c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:40.253043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2aa4a-2068-4861-8342-6b11745bc9c8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:40.276521 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1308/2613] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:40 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:40.280830 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee86dd-94d0-402c-b544-cb965ce1a1d3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:40.292618 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee86dd-94d0-402c-b544-cb965ce1a1d3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:40.298540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ff431-87c3-44d8-981d-122f4620a9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:40.299151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ff431-87c3-44d8-981d-122f4620a9ad admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.299422 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ff431-87c3-44d8-981d-122f4620a9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.299690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ff431-87c3-44d8-981d-122f4620a9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.300354 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ff431-87c3-44d8-981d-122f4620a9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:40.301996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ff431-87c3-44d8-981d-122f4620a9ad admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:40.312882 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1306/2614] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:40.317207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2aa4a-2068-4861-8342-6b11745bc9c8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:40.327847 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2aa4a-2068-4861-8342-6b11745bc9c8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:40.333915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e2c3c-bb6b-41ac-a4b2-48bc6e43e424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:40.334531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e2c3c-bb6b-41ac-a4b2-48bc6e43e424 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.334798 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e2c3c-bb6b-41ac-a4b2-48bc6e43e424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.335061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e2c3c-bb6b-41ac-a4b2-48bc6e43e424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.335703 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e2c3c-bb6b-41ac-a4b2-48bc6e43e424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:40.337325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e2c3c-bb6b-41ac-a4b2-48bc6e43e424 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:40.348413 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1309/2615] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:40.352797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ff431-87c3-44d8-981d-122f4620a9ad admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:40.363889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ff431-87c3-44d8-981d-122f4620a9ad admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:40.368926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e21bb3-99e9-48d6-8543-9aa734e7ebbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:40.369567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e21bb3-99e9-48d6-8543-9aa734e7ebbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.369838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e21bb3-99e9-48d6-8543-9aa734e7ebbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.370101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e21bb3-99e9-48d6-8543-9aa734e7ebbe admin admin] PATH_INFO: `/v3/projects/af5e2c1421944b87b001d4851d66ba3c/users/8898f3f431f24ab8a866a864d1146620/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.385092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e21bb3-99e9-48d6-8543-9aa734e7ebbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=af5e2c1421944b87b001d4851d66ba3c, user_id=8898f3f431f24ab8a866a864d1146620, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:40.387701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e21bb3-99e9-48d6-8543-9aa734e7ebbe admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:40.405661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1307/2616] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:40 2022] PUT /identity/v3/projects/af5e2c1421944b87b001d4851d66ba3c/users/8898f3f431f24ab8a866a864d1146620/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:26:40.420565 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e598d7-63e6-4919-8e9f-b949ffccc8f8 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.420917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e598d7-63e6-4919-8e9f-b949ffccc8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.421190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e598d7-63e6-4919-8e9f-b949ffccc8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.430322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:40.451545 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-33e598d7-63e6-4919-8e9f-b949ffccc8f8 None None] MFA Rules not processed for user `8898f3f431f24ab8a866a864d1146620`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:40.452920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-33e598d7-63e6-4919-8e9f-b949ffccc8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:40.525748 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1310/2617] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:40.538248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e21bb3-99e9-48d6-8543-9aa734e7ebbe admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:40.540304 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-76e21bb3-99e9-48d6-8543-9aa734e7ebbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:40.602413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e21bb3-99e9-48d6-8543-9aa734e7ebbe admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:40.606943 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7962ab-a884-456c-ac19-a973d0e54fbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:40.607636 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7962ab-a884-456c-ac19-a973d0e54fbb service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:40.607917 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7962ab-a884-456c-ac19-a973d0e54fbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:40.608188 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7962ab-a884-456c-ac19-a973d0e54fbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:40.617564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7962ab-a884-456c-ac19-a973d0e54fbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:40.618986 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7962ab-a884-456c-ac19-a973d0e54fbb service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:40.628697 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1308/2618] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:43.754882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e598d7-63e6-4919-8e9f-b949ffccc8f8 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:43.823309 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e598d7-63e6-4919-8e9f-b949ffccc8f8 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:43.830667 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:43.831307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:43.831588 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:43.831853 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] PATH_INFO: `/v3/users/8898f3f431f24ab8a866a864d1146620` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:43.833059 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8898f3f431f24ab8a866a864d1146620)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:43.835009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:43.891343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8898f3f431f24ab8a866a864d1146620'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:43.892110 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8898f3f431f24ab8a866a864d1146620'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:43.899805 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8898f3f431f24ab8a866a864d1146620'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:43.912448 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8898f3f431f24ab8a866a864d1146620'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:43.913149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8898f3f431f24ab8a866a864d1146620'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:43.927728 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1311/2619] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:26:43 2022] DELETE /identity/v3/users/8898f3f431f24ab8a866a864d1146620 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:43.932374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e87fd-332f-406c-a1a8-dd1b3e5c357e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:43.932892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e87fd-332f-406c-a1a8-dd1b3e5c357e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:43.933010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e87fd-332f-406c-a1a8-dd1b3e5c357e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:43.955804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-038e87fd-332f-406c-a1a8-dd1b3e5c357e None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:43.957077 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-038e87fd-332f-406c-a1a8-dd1b3e5c357e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:43.976118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:44.018782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1309/2620] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:44.029087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:44.031338 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:44.049168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:44.096976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a3f82-2bec-4673-89f3-4f8b80c224d3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:44.100792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d2690-341e-4633-992b-345bcc3c4643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:44.101394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2690-341e-4633-992b-345bcc3c4643 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:44.101666 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2690-341e-4633-992b-345bcc3c4643 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:44.101931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2690-341e-4633-992b-345bcc3c4643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:44.112041 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d2690-341e-4633-992b-345bcc3c4643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:44.113500 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d2690-341e-4633-992b-345bcc3c4643 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:44.123396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1312/2621] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:44.403013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e87fd-332f-406c-a1a8-dd1b3e5c357e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:44.404465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-038e87fd-332f-406c-a1a8-dd1b3e5c357e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:44.417418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e87fd-332f-406c-a1a8-dd1b3e5c357e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:44.422849 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a4803-1937-4853-af67-6c2bf86ffe77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:44.423788 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a4803-1937-4853-af67-6c2bf86ffe77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:44.424059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a4803-1937-4853-af67-6c2bf86ffe77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:44.424344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a4803-1937-4853-af67-6c2bf86ffe77 admin admin] PATH_INFO: `/v3/projects/af5e2c1421944b87b001d4851d66ba3c` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:44.425612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a4803-1937-4853-af67-6c2bf86ffe77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af5e2c1421944b87b001d4851d66ba3c)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:44.427526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a4803-1937-4853-af67-6c2bf86ffe77 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:44.608311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-554a4803-1937-4853-af67-6c2bf86ffe77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af5e2c1421944b87b001d4851d66ba3c', 'request_id': 'req-554a4803-1937-4853-af67-6c2bf86ffe77'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:44.616680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-554a4803-1937-4853-af67-6c2bf86ffe77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af5e2c1421944b87b001d4851d66ba3c', 'request_id': 'req-554a4803-1937-4853-af67-6c2bf86ffe77'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:44.624737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-554a4803-1937-4853-af67-6c2bf86ffe77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af5e2c1421944b87b001d4851d66ba3c', 'request_id': 'req-554a4803-1937-4853-af67-6c2bf86ffe77'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:44.639439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-554a4803-1937-4853-af67-6c2bf86ffe77 admin admin] The token cache is being invalidate because project af5e2c1421944b87b001d4851d66ba3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:26:44.639723 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-554a4803-1937-4853-af67-6c2bf86ffe77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:44.640332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-554a4803-1937-4853-af67-6c2bf86ffe77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:26:44.642011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1310/2622] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:26:44 2022] DELETE /identity/v3/projects/af5e2c1421944b87b001d4851d66ba3c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:44.647116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9837024-01ac-4fee-8d6f-d26cbb8a00d2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:44.647401 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9837024-01ac-4fee-8d6f-d26cbb8a00d2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:44.647672 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9837024-01ac-4fee-8d6f-d26cbb8a00d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:44.674409 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f9837024-01ac-4fee-8d6f-d26cbb8a00d2 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:44.675621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f9837024-01ac-4fee-8d6f-d26cbb8a00d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:44.740097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1313/2623] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:44.748871 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420d46a-b4f1-4e46-b691-a81334218a37 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:44.749144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420d46a-b4f1-4e46-b691-a81334218a37 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:44.749418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420d46a-b4f1-4e46-b691-a81334218a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:44.770894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-6420d46a-b4f1-4e46-b691-a81334218a37 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:44.772099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-6420d46a-b4f1-4e46-b691-a81334218a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:44.781110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1311/2624] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:44.786249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9837024-01ac-4fee-8d6f-d26cbb8a00d2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:44.796557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9837024-01ac-4fee-8d6f-d26cbb8a00d2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:44.802022 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402e58a-a097-47fa-8002-ca45591a7ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:44.802690 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e58a-a097-47fa-8002-ca45591a7ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:44.802965 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e58a-a097-47fa-8002-ca45591a7ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:44.803235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402e58a-a097-47fa-8002-ca45591a7ed9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:44.803892 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402e58a-a097-47fa-8002-ca45591a7ed9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:26:44.804196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402e58a-a097-47fa-8002-ca45591a7ed9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:44.805856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402e58a-a097-47fa-8002-ca45591a7ed9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:44.818225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1314/2625] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:26:44.822691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420d46a-b4f1-4e46-b691-a81334218a37 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:44.832837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420d46a-b4f1-4e46-b691-a81334218a37 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:44.837801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f2186-1751-4327-ab35-73d52b267aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:44.838393 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f2186-1751-4327-ab35-73d52b267aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:44.838651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f2186-1751-4327-ab35-73d52b267aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:44.838928 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f2186-1751-4327-ab35-73d52b267aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:44.839568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f2186-1751-4327-ab35-73d52b267aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:44.841479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f2186-1751-4327-ab35-73d52b267aa0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:44.870752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1312/2626] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:26:44 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:44.875468 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402e58a-a097-47fa-8002-ca45591a7ed9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:44.886082 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402e58a-a097-47fa-8002-ca45591a7ed9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:44.950413 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d79959-c3c0-40be-afdf-7b4372403f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:44.951018 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d79959-c3c0-40be-afdf-7b4372403f65 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:44.951293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d79959-c3c0-40be-afdf-7b4372403f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:44.951556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d79959-c3c0-40be-afdf-7b4372403f65 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:44.952172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d79959-c3c0-40be-afdf-7b4372403f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:44.954127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d79959-c3c0-40be-afdf-7b4372403f65 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:44.978751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1315/2627] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:26:44 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:26:44.983001 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f2186-1751-4327-ab35-73d52b267aa0 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:44.997765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f2186-1751-4327-ab35-73d52b267aa0 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:45.004373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82c07-9a1e-4cae-8f6f-15c505d2709d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:45.005039 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82c07-9a1e-4cae-8f6f-15c505d2709d admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:45.005298 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82c07-9a1e-4cae-8f6f-15c505d2709d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:45.005565 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de82c07-9a1e-4cae-8f6f-15c505d2709d admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:45.006217 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de82c07-9a1e-4cae-8f6f-15c505d2709d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:45.007804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de82c07-9a1e-4cae-8f6f-15c505d2709d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:45.023539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1313/2628] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:45.028551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d79959-c3c0-40be-afdf-7b4372403f65 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:45.043032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d79959-c3c0-40be-afdf-7b4372403f65 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:45.051754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe59d8-f159-4018-81f8-d347a7d43c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:45.052467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe59d8-f159-4018-81f8-d347a7d43c5d admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:45.052879 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe59d8-f159-4018-81f8-d347a7d43c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:45.053209 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe59d8-f159-4018-81f8-d347a7d43c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:45.054151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe59d8-f159-4018-81f8-d347a7d43c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:45.056565 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe59d8-f159-4018-81f8-d347a7d43c5d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:45.070818 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1316/2629] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:26:45.076835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82c07-9a1e-4cae-8f6f-15c505d2709d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:45.092041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de82c07-9a1e-4cae-8f6f-15c505d2709d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:45.099241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9155308-a6a7-45e8-ba0c-8de56515c4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:45.099884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9155308-a6a7-45e8-ba0c-8de56515c4bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:45.100160 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9155308-a6a7-45e8-ba0c-8de56515c4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:45.100413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9155308-a6a7-45e8-ba0c-8de56515c4bd admin admin] PATH_INFO: `/v3/projects/3c37b296273040cdb8c2a1281d91db03/users/b423b3a26d7640fa8e23f104cfeaecd0/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:45.117178 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9155308-a6a7-45e8-ba0c-8de56515c4bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c37b296273040cdb8c2a1281d91db03, user_id=b423b3a26d7640fa8e23f104cfeaecd0, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:45.119798 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9155308-a6a7-45e8-ba0c-8de56515c4bd admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:45.135861 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1314/2630] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:26:45 2022] PUT /identity/v3/projects/3c37b296273040cdb8c2a1281d91db03/users/b423b3a26d7640fa8e23f104cfeaecd0/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:26:45.205387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955db33c-09bc-4558-8582-d3800ea34356 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:45.205781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955db33c-09bc-4558-8582-d3800ea34356 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:45.206137 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955db33c-09bc-4558-8582-d3800ea34356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:45.252300 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-955db33c-09bc-4558-8582-d3800ea34356 None None] MFA Rules not processed for user `b423b3a26d7640fa8e23f104cfeaecd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:45.254540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-955db33c-09bc-4558-8582-d3800ea34356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:45.264772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:45.331255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1317/2631] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:45.343830 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9155308-a6a7-45e8-ba0c-8de56515c4bd admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:45.345760 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d9155308-a6a7-45e8-ba0c-8de56515c4bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:45.401280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:45.414780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9155308-a6a7-45e8-ba0c-8de56515c4bd admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:45.419162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512d4a2-bc79-4b72-a61f-f74e90b46859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:45.419773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512d4a2-bc79-4b72-a61f-f74e90b46859 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:45.420042 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512d4a2-bc79-4b72-a61f-f74e90b46859 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:45.420303 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512d4a2-bc79-4b72-a61f-f74e90b46859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:45.430057 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512d4a2-bc79-4b72-a61f-f74e90b46859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:45.431545 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512d4a2-bc79-4b72-a61f-f74e90b46859 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:45.440522 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1315/2632] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:46.833979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeabeaf-d066-49d6-8b8b-0b457583f064 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:46.834547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeabeaf-d066-49d6-8b8b-0b457583f064 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:46.834547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeabeaf-d066-49d6-8b8b-0b457583f064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:46.855184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-ffeabeaf-d066-49d6-8b8b-0b457583f064 None None] MFA Rules not processed for user `b423b3a26d7640fa8e23f104cfeaecd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:26:46.856398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ffeabeaf-d066-49d6-8b8b-0b457583f064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:46.866874 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1318/2633] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:26:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:26:49.287916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512d4a2-bc79-4b72-a61f-f74e90b46859 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:49.289774 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f512d4a2-bc79-4b72-a61f-f74e90b46859 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:49.358900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512d4a2-bc79-4b72-a61f-f74e90b46859 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:49.363237 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b48584-35b9-4f03-b55e-423ac6ae72b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:49.364410 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b48584-35b9-4f03-b55e-423ac6ae72b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:49.364410 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b48584-35b9-4f03-b55e-423ac6ae72b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:49.364410 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b48584-35b9-4f03-b55e-423ac6ae72b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:49.366299 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d8b48584-35b9-4f03-b55e-423ac6ae72b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:49.437934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b48584-35b9-4f03-b55e-423ac6ae72b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:49.439705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b48584-35b9-4f03-b55e-423ac6ae72b2 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:49.450989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1316/2634] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:56.848658 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=25 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:26:58.096123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeabeaf-d066-49d6-8b8b-0b457583f064 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:26:58.107233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeabeaf-d066-49d6-8b8b-0b457583f064 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:26:58.111085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b3c93-83fd-42f0-9d21-5b082f758e6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:26:58.111717 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b3c93-83fd-42f0-9d21-5b082f758e6a service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:26:58.111991 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b3c93-83fd-42f0-9d21-5b082f758e6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:26:58.112257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b3c93-83fd-42f0-9d21-5b082f758e6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:26:58.114066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3a9b3c93-83fd-42f0-9d21-5b082f758e6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:26:58.189158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b3c93-83fd-42f0-9d21-5b082f758e6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:26:58.190607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b3c93-83fd-42f0-9d21-5b082f758e6a service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:26:58.201268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1319/2635] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:26:59.446024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=54 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:01.678531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b48584-35b9-4f03-b55e-423ac6ae72b2 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:01.680295 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d8b48584-35b9-4f03-b55e-423ac6ae72b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:01.764529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b48584-35b9-4f03-b55e-423ac6ae72b2 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:01.770123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f3b2b-cd90-428f-a48e-c8abf2602b8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:01.770726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3b2b-cd90-428f-a48e-c8abf2602b8f service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:01.771006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3b2b-cd90-428f-a48e-c8abf2602b8f service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:01.771256 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f3b2b-cd90-428f-a48e-c8abf2602b8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:01.773170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a60f3b2b-cd90-428f-a48e-c8abf2602b8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:01.785428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f3b2b-cd90-428f-a48e-c8abf2602b8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:01.786996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f3b2b-cd90-428f-a48e-c8abf2602b8f service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:01.797131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1317/2636] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:02.160998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595f357-f03f-459f-abbd-2266a0dd6683 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:02.161555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595f357-f03f-459f-abbd-2266a0dd6683 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:02.161555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595f357-f03f-459f-abbd-2266a0dd6683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:02.181756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-5595f357-f03f-459f-abbd-2266a0dd6683 None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:02.183084 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5595f357-f03f-459f-abbd-2266a0dd6683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:02.248451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1320/2637] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:27:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:02.260450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f3b2b-cd90-428f-a48e-c8abf2602b8f service placement] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:02.262088 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a60f3b2b-cd90-428f-a48e-c8abf2602b8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:02.331136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f3b2b-cd90-428f-a48e-c8abf2602b8f service placement] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:02.336554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6815a81-10c0-44d3-b8fb-5c54299b745a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:02.338368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6815a81-10c0-44d3-b8fb-5c54299b745a service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:02.338368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6815a81-10c0-44d3-b8fb-5c54299b745a service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:02.338368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6815a81-10c0-44d3-b8fb-5c54299b745a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:02.346872 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6815a81-10c0-44d3-b8fb-5c54299b745a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:02.348442 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6815a81-10c0-44d3-b8fb-5c54299b745a service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:02.357062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1318/2638] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:27:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:27:02.959849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e724c-f157-41c7-bdae-803849063c4e None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:02.960212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e724c-f157-41c7-bdae-803849063c4e None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:02.960608 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e724c-f157-41c7-bdae-803849063c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:02.981176 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d03e724c-f157-41c7-bdae-803849063c4e None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:02.982431 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d03e724c-f157-41c7-bdae-803849063c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:02.989917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1321/2639] 10.222.0.75 () {58 vars in 967 bytes} [Fri Sep 23 15:27:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:03.002740 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6815a81-10c0-44d3-b8fb-5c54299b745a service swift] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:03.014251 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6815a81-10c0-44d3-b8fb-5c54299b745a service swift] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:03.019120 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f136dd-fa7a-47d8-b947-9017859fe694 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:03.019828 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f136dd-fa7a-47d8-b947-9017859fe694 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:03.020102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f136dd-fa7a-47d8-b947-9017859fe694 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:03.020362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f136dd-fa7a-47d8-b947-9017859fe694 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:03.028581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f136dd-fa7a-47d8-b947-9017859fe694 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:03.030208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f136dd-fa7a-47d8-b947-9017859fe694 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:03.038526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1319/2640] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:27:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:27:03.316533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c831a-58f3-4508-86e5-78a967563c08 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:03.317096 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c831a-58f3-4508-86e5-78a967563c08 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:03.317246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c831a-58f3-4508-86e5-78a967563c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:03.339148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-869c831a-58f3-4508-86e5-78a967563c08 None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:03.340135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-869c831a-58f3-4508-86e5-78a967563c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:03.348556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1322/2641] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:27:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:03.362191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f136dd-fa7a-47d8-b947-9017859fe694 service swift] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:03.374135 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f136dd-fa7a-47d8-b947-9017859fe694 service swift] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:03.380672 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19134f11-6cd1-4349-91b2-1c66849509a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:03.381280 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19134f11-6cd1-4349-91b2-1c66849509a8 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:03.381548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19134f11-6cd1-4349-91b2-1c66849509a8 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:03.381809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19134f11-6cd1-4349-91b2-1c66849509a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:03.391115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19134f11-6cd1-4349-91b2-1c66849509a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:03.392620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19134f11-6cd1-4349-91b2-1c66849509a8 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:03.401628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1320/2642] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:27:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:27:03.780635 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a682e6-8307-4590-8c9e-861eb74d04ae None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:03.781274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a682e6-8307-4590-8c9e-861eb74d04ae None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:03.781541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a682e6-8307-4590-8c9e-861eb74d04ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:03.807543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-63a682e6-8307-4590-8c9e-861eb74d04ae None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:03.808882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-63a682e6-8307-4590-8c9e-861eb74d04ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:03.817610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1323/2643] 10.222.0.75 () {58 vars in 967 bytes} [Fri Sep 23 15:27:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:03.831146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19134f11-6cd1-4349-91b2-1c66849509a8 service swift] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:03.843988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19134f11-6cd1-4349-91b2-1c66849509a8 service swift] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:03.849526 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae1a51-f1b0-4577-87e9-b1d5161916a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:03.850240 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae1a51-f1b0-4577-87e9-b1d5161916a8 service swift] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:03.850434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae1a51-f1b0-4577-87e9-b1d5161916a8 service swift] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:03.850701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae1a51-f1b0-4577-87e9-b1d5161916a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:03.860035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae1a51-f1b0-4577-87e9-b1d5161916a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:03.861516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae1a51-f1b0-4577-87e9-b1d5161916a8 service swift] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:03.870225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1321/2644] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:27:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:27:05.111317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a682e6-8307-4590-8c9e-861eb74d04ae None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:05.113495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-63a682e6-8307-4590-8c9e-861eb74d04ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:05.151554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a682e6-8307-4590-8c9e-861eb74d04ae None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:05.154793 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:05.155421 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:05.155692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:05.155955 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] PATH_INFO: `/v3/users/3077d51be16e40d9bb70b984583c0f8a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:05.157101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3077d51be16e40d9bb70b984583c0f8a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:05.158998 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:05.213355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3077d51be16e40d9bb70b984583c0f8a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.214112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3077d51be16e40d9bb70b984583c0f8a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.222108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3077d51be16e40d9bb70b984583c0f8a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.235293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3077d51be16e40d9bb70b984583c0f8a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.236053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3077d51be16e40d9bb70b984583c0f8a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.243911 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:05.256517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1324/2645] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:27:05 2022] DELETE /identity/v3/users/3077d51be16e40d9bb70b984583c0f8a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:05.431837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae1a51-f1b0-4577-87e9-b1d5161916a8 service swift] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:05.444665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae1a51-f1b0-4577-87e9-b1d5161916a8 service swift] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:05.485651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:05.504875 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:05.505504 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:05.505779 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:05.506048 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] PATH_INFO: `/v3/projects/592a763d7123405fb0576b00edfcd1bf` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:05.507196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=592a763d7123405fb0576b00edfcd1bf)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:05.509178 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:05.562049 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '592a763d7123405fb0576b00edfcd1bf', 'request_id': 'req-4bda0a60-bdc1-49a7-8f66-0182a05da108'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.568559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '592a763d7123405fb0576b00edfcd1bf', 'request_id': 'req-4bda0a60-bdc1-49a7-8f66-0182a05da108'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.574709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '592a763d7123405fb0576b00edfcd1bf', 'request_id': 'req-4bda0a60-bdc1-49a7-8f66-0182a05da108'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.589769 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] The token cache is being invalidate because project 592a763d7123405fb0576b00edfcd1bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:05.590055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.590681 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.592206 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1322/2646] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:05 2022] DELETE /identity/v3/projects/592a763d7123405fb0576b00edfcd1bf => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:05.596257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:05.597778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:05.611308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a86aa-ad9f-4046-9f04-987e3b3ddea7 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:05.671360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:05.671987 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:05.672262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:05.672523 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] PATH_INFO: `/v3/users/7081e82606914f3b975cdccaddf2088f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:05.673721 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7081e82606914f3b975cdccaddf2088f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:05.675647 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:05.725988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7081e82606914f3b975cdccaddf2088f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.726567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7081e82606914f3b975cdccaddf2088f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.733638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7081e82606914f3b975cdccaddf2088f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.746682 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7081e82606914f3b975cdccaddf2088f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.747789 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7081e82606914f3b975cdccaddf2088f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:05.767495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1325/2647] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:27:05 2022] DELETE /identity/v3/users/7081e82606914f3b975cdccaddf2088f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:05.951612 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:05.953123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:05.970429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda0a60-bdc1-49a7-8f66-0182a05da108 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:06.032227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d05d43-f0f9-414f-a75b-cafcd35b1eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:06.032894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d05d43-f0f9-414f-a75b-cafcd35b1eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:06.033164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d05d43-f0f9-414f-a75b-cafcd35b1eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:06.033433 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d05d43-f0f9-414f-a75b-cafcd35b1eba admin admin] PATH_INFO: `/v3/projects/d965e9d40a02427aba973b31153c16a6` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:06.034603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d05d43-f0f9-414f-a75b-cafcd35b1eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=d965e9d40a02427aba973b31153c16a6)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:06.036496 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d05d43-f0f9-414f-a75b-cafcd35b1eba admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:06.085279 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-32d05d43-f0f9-414f-a75b-cafcd35b1eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd965e9d40a02427aba973b31153c16a6', 'request_id': 'req-32d05d43-f0f9-414f-a75b-cafcd35b1eba'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.091620 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-32d05d43-f0f9-414f-a75b-cafcd35b1eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd965e9d40a02427aba973b31153c16a6', 'request_id': 'req-32d05d43-f0f9-414f-a75b-cafcd35b1eba'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.097073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-32d05d43-f0f9-414f-a75b-cafcd35b1eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd965e9d40a02427aba973b31153c16a6', 'request_id': 'req-32d05d43-f0f9-414f-a75b-cafcd35b1eba'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.108262 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:06.112241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-32d05d43-f0f9-414f-a75b-cafcd35b1eba admin admin] The token cache is being invalidate because project d965e9d40a02427aba973b31153c16a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:06.112675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-32d05d43-f0f9-414f-a75b-cafcd35b1eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.113548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-32d05d43-f0f9-414f-a75b-cafcd35b1eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.115059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1323/2648] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:05 2022] DELETE /identity/v3/projects/d965e9d40a02427aba973b31153c16a6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:06.169390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:06.171182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:06.185579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019a10f-cec0-4717-b231-02eadc5c38d5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:06.255539 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:06.256104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:06.256385 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:06.256686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] PATH_INFO: `/v3/users/b423b3a26d7640fa8e23f104cfeaecd0` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:06.257869 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b423b3a26d7640fa8e23f104cfeaecd0)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:06.259848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:06.273854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:06.317442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b423b3a26d7640fa8e23f104cfeaecd0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.318162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b423b3a26d7640fa8e23f104cfeaecd0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.327634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b423b3a26d7640fa8e23f104cfeaecd0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.343142 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b423b3a26d7640fa8e23f104cfeaecd0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.343981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b423b3a26d7640fa8e23f104cfeaecd0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.363662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1326/2649] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:27:06 2022] DELETE /identity/v3/users/b423b3a26d7640fa8e23f104cfeaecd0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:06.366712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9c480-b7fc-4b97-852f-27b6b96f11c3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:06.367301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9c480-b7fc-4b97-852f-27b6b96f11c3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:06.367800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9c480-b7fc-4b97-852f-27b6b96f11c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:06.392914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-03f9c480-b7fc-4b97-852f-27b6b96f11c3 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:06.394590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-03f9c480-b7fc-4b97-852f-27b6b96f11c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:06.463969 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1324/2650] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:06.474126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:06.475632 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:06.536218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd6e9f-5732-4d7a-9ade-67b7ab828735 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:06.539975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949c18d-3bcb-47c7-8e0f-2ddb7ebbd27a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:06.540808 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949c18d-3bcb-47c7-8e0f-2ddb7ebbd27a service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:06.541088 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949c18d-3bcb-47c7-8e0f-2ddb7ebbd27a service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:06.541513 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949c18d-3bcb-47c7-8e0f-2ddb7ebbd27a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:06.551609 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949c18d-3bcb-47c7-8e0f-2ddb7ebbd27a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:06.552958 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949c18d-3bcb-47c7-8e0f-2ddb7ebbd27a service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:06.563395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1327/2651] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:06.738234 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9c480-b7fc-4b97-852f-27b6b96f11c3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:06.739859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-03f9c480-b7fc-4b97-852f-27b6b96f11c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:06.753786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9c480-b7fc-4b97-852f-27b6b96f11c3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:06.760150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:06.760841 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:06.761112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:06.761399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea admin admin] PATH_INFO: `/v3/projects/3c37b296273040cdb8c2a1281d91db03` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:06.762532 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c37b296273040cdb8c2a1281d91db03)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:06.764411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:06.815061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c37b296273040cdb8c2a1281d91db03', 'request_id': 'req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.821396 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c37b296273040cdb8c2a1281d91db03', 'request_id': 'req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.827533 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c37b296273040cdb8c2a1281d91db03', 'request_id': 'req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.841998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea admin admin] The token cache is being invalidate because project 3c37b296273040cdb8c2a1281d91db03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:27:06.842268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.842890 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b2b19ffb-7d66-4add-8293-3a5fb3aca1ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:27:06.844378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1325/2652] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:27:06 2022] DELETE /identity/v3/projects/3c37b296273040cdb8c2a1281d91db03 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:27:06.849483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b524bcf-d81b-41fb-8875-b7a7e0bedcf4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:06.849791 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b524bcf-d81b-41fb-8875-b7a7e0bedcf4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:06.850078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b524bcf-d81b-41fb-8875-b7a7e0bedcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:06.871670 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0b524bcf-d81b-41fb-8875-b7a7e0bedcf4 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:06.873240 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0b524bcf-d81b-41fb-8875-b7a7e0bedcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:06.936225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1328/2653] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:06.944229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef18097-4f44-46c9-97fa-69a2dacb4ab1 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:06.944506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef18097-4f44-46c9-97fa-69a2dacb4ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:06.944803 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef18097-4f44-46c9-97fa-69a2dacb4ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:06.965885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-fef18097-4f44-46c9-97fa-69a2dacb4ab1 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:06.967083 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fef18097-4f44-46c9-97fa-69a2dacb4ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:06.975900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1326/2654] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:06.980377 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b524bcf-d81b-41fb-8875-b7a7e0bedcf4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:06.990105 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b524bcf-d81b-41fb-8875-b7a7e0bedcf4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:06.995155 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3cc63-e146-4326-bbe6-8285d855ce26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:06.995729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3cc63-e146-4326-bbe6-8285d855ce26 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:06.996003 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3cc63-e146-4326-bbe6-8285d855ce26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:06.996264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3cc63-e146-4326-bbe6-8285d855ce26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:06.997287 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3cc63-e146-4326-bbe6-8285d855ce26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:27:06.997577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3cc63-e146-4326-bbe6-8285d855ce26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:06.999175 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb3cc63-e146-4326-bbe6-8285d855ce26 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:07.011639 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1329/2655] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:27:07.016218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef18097-4f44-46c9-97fa-69a2dacb4ab1 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:07.026374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef18097-4f44-46c9-97fa-69a2dacb4ab1 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:07.031862 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fe6cf-4aa0-4897-8692-cb00f4beac90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:07.032451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fe6cf-4aa0-4897-8692-cb00f4beac90 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:07.032763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fe6cf-4aa0-4897-8692-cb00f4beac90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:07.033028 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fe6cf-4aa0-4897-8692-cb00f4beac90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:07.033666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fe6cf-4aa0-4897-8692-cb00f4beac90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:07.035524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fe6cf-4aa0-4897-8692-cb00f4beac90 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:07.064567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1327/2656] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:27:07 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:07.068902 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3cc63-e146-4326-bbe6-8285d855ce26 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:07.079683 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3cc63-e146-4326-bbe6-8285d855ce26 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:07.098285 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:07.147669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec69ed5-0afb-4aa7-960a-e07c4805add1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:07.148336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec69ed5-0afb-4aa7-960a-e07c4805add1 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:07.148734 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec69ed5-0afb-4aa7-960a-e07c4805add1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:07.149069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec69ed5-0afb-4aa7-960a-e07c4805add1 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:07.150251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec69ed5-0afb-4aa7-960a-e07c4805add1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:07.152434 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec69ed5-0afb-4aa7-960a-e07c4805add1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:07.175971 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1330/2657] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:27:07 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:07.180257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fe6cf-4aa0-4897-8692-cb00f4beac90 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:07.191368 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fe6cf-4aa0-4897-8692-cb00f4beac90 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:07.197266 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b444b8-d935-4592-b311-de3e0b565383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:07.198163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b444b8-d935-4592-b311-de3e0b565383 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:07.198434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b444b8-d935-4592-b311-de3e0b565383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:07.198696 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b444b8-d935-4592-b311-de3e0b565383 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:07.199340 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b444b8-d935-4592-b311-de3e0b565383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:07.200940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b444b8-d935-4592-b311-de3e0b565383 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:07.211547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1328/2658] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:07.216461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec69ed5-0afb-4aa7-960a-e07c4805add1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:07.232123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec69ed5-0afb-4aa7-960a-e07c4805add1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:07.241071 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87027e03-aadb-4f5b-b890-7f6bab8dc14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:07.241919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87027e03-aadb-4f5b-b890-7f6bab8dc14a admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:07.242263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87027e03-aadb-4f5b-b890-7f6bab8dc14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:07.242615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87027e03-aadb-4f5b-b890-7f6bab8dc14a admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:07.243570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87027e03-aadb-4f5b-b890-7f6bab8dc14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:07.245452 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87027e03-aadb-4f5b-b890-7f6bab8dc14a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:07.257006 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1331/2659] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:07.261384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b444b8-d935-4592-b311-de3e0b565383 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:07.271866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b444b8-d935-4592-b311-de3e0b565383 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:07.278198 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e08d8-777d-4fea-82e6-521915793617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:07.278846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e08d8-777d-4fea-82e6-521915793617 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:07.279116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e08d8-777d-4fea-82e6-521915793617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:07.279383 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e08d8-777d-4fea-82e6-521915793617 admin admin] PATH_INFO: `/v3/projects/c05aa308acb842198935488c659d9ab3/users/cd5b918b0427408d9a70ca568d75ad58/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:07.299097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e08d8-777d-4fea-82e6-521915793617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c05aa308acb842198935488c659d9ab3, user_id=cd5b918b0427408d9a70ca568d75ad58, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:07.303299 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e08d8-777d-4fea-82e6-521915793617 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:07.323064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1329/2660] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:07 2022] PUT /identity/v3/projects/c05aa308acb842198935488c659d9ab3/users/cd5b918b0427408d9a70ca568d75ad58/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:27:07.328193 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85a6ab-8468-4813-8d29-136262102992 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:07.328477 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85a6ab-8468-4813-8d29-136262102992 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:07.328774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85a6ab-8468-4813-8d29-136262102992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:07.347848 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-de85a6ab-8468-4813-8d29-136262102992 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:07.349097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-de85a6ab-8468-4813-8d29-136262102992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:07.410576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1332/2661] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:07.420908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e08d8-777d-4fea-82e6-521915793617 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:07.422703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-491e08d8-777d-4fea-82e6-521915793617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:07.444683 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:07.481926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e08d8-777d-4fea-82e6-521915793617 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:07.486161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01973e4-69fd-490c-b8a3-ad49aa4858f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:07.486763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01973e4-69fd-490c-b8a3-ad49aa4858f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:07.487030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01973e4-69fd-490c-b8a3-ad49aa4858f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:07.487289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01973e4-69fd-490c-b8a3-ad49aa4858f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:07.498216 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01973e4-69fd-490c-b8a3-ad49aa4858f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:07.499674 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01973e4-69fd-490c-b8a3-ad49aa4858f3 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:07.509444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1330/2662] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:11.556981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0499c-3561-4887-a0fc-57a8451d76bc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:11.558202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0499c-3561-4887-a0fc-57a8451d76bc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:11.558664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0499c-3561-4887-a0fc-57a8451d76bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:11.589078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-34c0499c-3561-4887-a0fc-57a8451d76bc None None] MFA Rules not processed for user `cd5b918b0427408d9a70ca568d75ad58`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:11.590241 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-34c0499c-3561-4887-a0fc-57a8451d76bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:11.663391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1333/2663] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:27:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:11.668337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01973e4-69fd-490c-b8a3-ad49aa4858f3 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:11.680145 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01973e4-69fd-490c-b8a3-ad49aa4858f3 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:11.685464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7096a20-4347-4ace-8778-ab69196f0d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:11.686062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7096a20-4347-4ace-8778-ab69196f0d6a admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:11.686334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7096a20-4347-4ace-8778-ab69196f0d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:11.686595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7096a20-4347-4ace-8778-ab69196f0d6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:11.687240 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7096a20-4347-4ace-8778-ab69196f0d6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:11.689163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7096a20-4347-4ace-8778-ab69196f0d6a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:11.719499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1331/2664] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:27:11 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:11.723447 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0499c-3561-4887-a0fc-57a8451d76bc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:11.754957 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0499c-3561-4887-a0fc-57a8451d76bc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:11.781931 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:11.817515 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275061a0-b38b-4cd6-ac2c-6c28836ea810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:11.818132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275061a0-b38b-4cd6-ac2c-6c28836ea810 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:11.818407 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275061a0-b38b-4cd6-ac2c-6c28836ea810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:11.818677 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275061a0-b38b-4cd6-ac2c-6c28836ea810 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:11.819318 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275061a0-b38b-4cd6-ac2c-6c28836ea810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:11.821322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275061a0-b38b-4cd6-ac2c-6c28836ea810 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:11.846610 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1334/2665] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:27:11 2022] POST /identity/v3/users => generated 296 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:27:11.850866 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7096a20-4347-4ace-8778-ab69196f0d6a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:11.864150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7096a20-4347-4ace-8778-ab69196f0d6a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:11.869788 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893573d4-3453-4f6d-8686-162447e3b2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:11.870528 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893573d4-3453-4f6d-8686-162447e3b2ed admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:11.870803 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893573d4-3453-4f6d-8686-162447e3b2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:11.871068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893573d4-3453-4f6d-8686-162447e3b2ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:11.871717 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893573d4-3453-4f6d-8686-162447e3b2ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:11.873709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893573d4-3453-4f6d-8686-162447e3b2ed admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:11.885604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1332/2666] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:27:11.890170 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275061a0-b38b-4cd6-ac2c-6c28836ea810 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:11.901042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275061a0-b38b-4cd6-ac2c-6c28836ea810 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:11.905941 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da218382-fb68-4791-93cd-16afed9edc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:11.906722 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da218382-fb68-4791-93cd-16afed9edc21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:11.907102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da218382-fb68-4791-93cd-16afed9edc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:11.907480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da218382-fb68-4791-93cd-16afed9edc21 admin admin] PATH_INFO: `/v3/projects/d47034a29e454f229fd8f2b851fbd546/users/cf3bbbbd7285451eb4cacc4e3713bf7a/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:11.923768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da218382-fb68-4791-93cd-16afed9edc21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d47034a29e454f229fd8f2b851fbd546, user_id=cf3bbbbd7285451eb4cacc4e3713bf7a, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:11.926484 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da218382-fb68-4791-93cd-16afed9edc21 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:11.948053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1335/2667] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:11 2022] PUT /identity/v3/projects/d47034a29e454f229fd8f2b851fbd546/users/cf3bbbbd7285451eb4cacc4e3713bf7a/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:27:11.949637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893573d4-3453-4f6d-8686-162447e3b2ed admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:11.960184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893573d4-3453-4f6d-8686-162447e3b2ed admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:12.020204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d39a89-1308-44ea-8659-d873dbf2b234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:12.020962 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d39a89-1308-44ea-8659-d873dbf2b234 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:12.021236 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d39a89-1308-44ea-8659-d873dbf2b234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:12.021509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d39a89-1308-44ea-8659-d873dbf2b234 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:12.022156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d39a89-1308-44ea-8659-d873dbf2b234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:12.023750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d39a89-1308-44ea-8659-d873dbf2b234 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:12.034598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1333/2668] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:27:12.038824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da218382-fb68-4791-93cd-16afed9edc21 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:12.049936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da218382-fb68-4791-93cd-16afed9edc21 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:12.054652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c485c-d97c-4cc2-b16e-ded5e5028546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:12.055324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c485c-d97c-4cc2-b16e-ded5e5028546 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:12.055600 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c485c-d97c-4cc2-b16e-ded5e5028546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:12.055918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c485c-d97c-4cc2-b16e-ded5e5028546 admin admin] PATH_INFO: `/v3/projects/d47034a29e454f229fd8f2b851fbd546/users/cf3bbbbd7285451eb4cacc4e3713bf7a/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:12.057962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c485c-d97c-4cc2-b16e-ded5e5028546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d47034a29e454f229fd8f2b851fbd546, user_id=cf3bbbbd7285451eb4cacc4e3713bf7a, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:12.060679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c485c-d97c-4cc2-b16e-ded5e5028546 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:12.071933 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1336/2669] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:27:12 2022] PUT /identity/v3/projects/d47034a29e454f229fd8f2b851fbd546/users/cf3bbbbd7285451eb4cacc4e3713bf7a/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:27:16.768408 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dadfaa-2769-4127-8af5-f05b368b72eb None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:16.768954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dadfaa-2769-4127-8af5-f05b368b72eb None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:16.768954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dadfaa-2769-4127-8af5-f05b368b72eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:16.801323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f9dadfaa-2769-4127-8af5-f05b368b72eb None None] MFA Rules not processed for user `cf3bbbbd7285451eb4cacc4e3713bf7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:16.802591 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f9dadfaa-2769-4127-8af5-f05b368b72eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:16.889092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1334/2670] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:27:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:16.905801 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c485c-d97c-4cc2-b16e-ded5e5028546 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:16.907743 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e39c485c-d97c-4cc2-b16e-ded5e5028546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:16.993884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c485c-d97c-4cc2-b16e-ded5e5028546 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:17.000262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc25bc-1069-4e72-a59a-f746af0ee874 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:17.000909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc25bc-1069-4e72-a59a-f746af0ee874 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:17.001185 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc25bc-1069-4e72-a59a-f746af0ee874 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:17.001441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc25bc-1069-4e72-a59a-f746af0ee874 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:17.062617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc25bc-1069-4e72-a59a-f746af0ee874 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:17.064343 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc25bc-1069-4e72-a59a-f746af0ee874 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:17.074337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1337/2671] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:27:22.432731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141514fa-dc81-4609-86d2-6296164e86dd None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:22.433208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141514fa-dc81-4609-86d2-6296164e86dd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:22.433324 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141514fa-dc81-4609-86d2-6296164e86dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:22.455394 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-141514fa-dc81-4609-86d2-6296164e86dd None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:22.456613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-141514fa-dc81-4609-86d2-6296164e86dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:22.462273 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:22.524438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1335/2672] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:27:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:22.552825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc25bc-1069-4e72-a59a-f746af0ee874 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:22.554741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-21dc25bc-1069-4e72-a59a-f746af0ee874 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:22.581823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:22.642183 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc25bc-1069-4e72-a59a-f746af0ee874 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:22.649346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af7531-382d-4921-9ebc-6bf82f12f180 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:22.649979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af7531-382d-4921-9ebc-6bf82f12f180 service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:22.650252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af7531-382d-4921-9ebc-6bf82f12f180 service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:22.650516 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af7531-382d-4921-9ebc-6bf82f12f180 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:22.659935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af7531-382d-4921-9ebc-6bf82f12f180 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:22.661572 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af7531-382d-4921-9ebc-6bf82f12f180 service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:22.669814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1338/2673] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:27:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 15:27:32.513966 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=20 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:32.666599 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=24 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:39.290489 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ace769-df35-4b5c-b4ac-c11467fbefcd None None] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:39.291129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ace769-df35-4b5c-b4ac-c11467fbefcd None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:39.291129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ace769-df35-4b5c-b4ac-c11467fbefcd None None] PATH_INFO: `/` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:39.292567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1336/2674] 10.222.0.75 () {54 vars in 921 bytes} [Fri Sep 23 15:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 15:27:39.297803 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245071a3-9037-4b4c-9a94-3a503be8ca71 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:39.298326 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245071a3-9037-4b4c-9a94-3a503be8ca71 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:39.298433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245071a3-9037-4b4c-9a94-3a503be8ca71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:39.319397 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-245071a3-9037-4b4c-9a94-3a503be8ca71 None None] MFA Rules not processed for user `2af1ba390fff41a68ff1a0035901c318`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:39.320720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-245071a3-9037-4b4c-9a94-3a503be8ca71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:39.331099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1339/2675] 10.222.0.75 () {58 vars in 1018 bytes} [Fri Sep 23 15:27:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:39.343503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ace769-df35-4b5c-b4ac-c11467fbefcd None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:39.345583 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-69ace769-df35-4b5c-b4ac-c11467fbefcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:39.406476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ace769-df35-4b5c-b4ac-c11467fbefcd None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:39.410543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6da427-b5de-40d4-9473-2280c9087c60 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac783fbe95ea4b74bd46af415c9734fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:39.411146 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6da427-b5de-40d4-9473-2280c9087c60 service glance] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:39.411412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6da427-b5de-40d4-9473-2280c9087c60 service glance] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:39.411667 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6da427-b5de-40d4-9473-2280c9087c60 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:39.422539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6da427-b5de-40d4-9473-2280c9087c60 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:39.424024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6da427-b5de-40d4-9473-2280c9087c60 service glance] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:39.434852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1337/2676] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:27:41.318790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb58277-d26b-4f12-81c7-238390b3c36b None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:41.319296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb58277-d26b-4f12-81c7-238390b3c36b None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:41.319389 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb58277-d26b-4f12-81c7-238390b3c36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:41.342187 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-feb58277-d26b-4f12-81c7-238390b3c36b None None] MFA Rules not processed for user `cd5b918b0427408d9a70ca568d75ad58`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:41.343386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-feb58277-d26b-4f12-81c7-238390b3c36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:41.350956 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1340/2677] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:27:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:41.467587 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419560d6-cb44-479d-92fd-5c7568766fcc None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:41.468131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419560d6-cb44-479d-92fd-5c7568766fcc None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:41.468131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419560d6-cb44-479d-92fd-5c7568766fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:41.487801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-419560d6-cb44-479d-92fd-5c7568766fcc None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:27:41.489041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-419560d6-cb44-479d-92fd-5c7568766fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:41.496778 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1338/2678] 10.222.0.75 () {58 vars in 1009 bytes} [Fri Sep 23 15:27:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:27:41.509696 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb58277-d26b-4f12-81c7-238390b3c36b None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:41.522065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb58277-d26b-4f12-81c7-238390b3c36b None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:41.527172 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d8966-6af7-450d-b8b2-83f78259c00b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:41.527876 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d8966-6af7-450d-b8b2-83f78259c00b service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:41.528218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d8966-6af7-450d-b8b2-83f78259c00b service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:41.528582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d8966-6af7-450d-b8b2-83f78259c00b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:41.537691 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d8966-6af7-450d-b8b2-83f78259c00b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:41.538932 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d8966-6af7-450d-b8b2-83f78259c00b service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:41.550078 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1341/2679] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:27:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 15:27:51.481994 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=29 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:51.546762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=12 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:27:58.616207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419560d6-cb44-479d-92fd-5c7568766fcc None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:27:58.671595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419560d6-cb44-479d-92fd-5c7568766fcc None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:27:58.676821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e37c3f-d87a-4fbf-b9a4-50de8b90710f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:27:58.677702 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e37c3f-d87a-4fbf-b9a4-50de8b90710f service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:27:58.678100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e37c3f-d87a-4fbf-b9a4-50de8b90710f service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:27:58.678517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e37c3f-d87a-4fbf-b9a4-50de8b90710f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:27:58.681135 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e7e37c3f-d87a-4fbf-b9a4-50de8b90710f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:27:58.701122 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e37c3f-d87a-4fbf-b9a4-50de8b90710f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:27:58.703726 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e37c3f-d87a-4fbf-b9a4-50de8b90710f service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:27:58.723934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1339/2680] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:27:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:00.375244 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d8966-6af7-450d-b8b2-83f78259c00b service swift] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:00.377064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-629d8966-6af7-450d-b8b2-83f78259c00b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:00.391054 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d8966-6af7-450d-b8b2-83f78259c00b service swift] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:00.395208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafb34a-3c09-4275-bfd3-6b026ea14436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:00.395796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafb34a-3c09-4275-bfd3-6b026ea14436 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:00.396067 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafb34a-3c09-4275-bfd3-6b026ea14436 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:00.396331 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafb34a-3c09-4275-bfd3-6b026ea14436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:00.398373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-7fafb34a-3c09-4275-bfd3-6b026ea14436 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:00.411738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafb34a-3c09-4275-bfd3-6b026ea14436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:00.413164 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fafb34a-3c09-4275-bfd3-6b026ea14436 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:00.424040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1342/2681] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:02.798073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34608a37-0a5d-4387-9e23-6142194f720b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:02.798631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34608a37-0a5d-4387-9e23-6142194f720b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:02.798631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34608a37-0a5d-4387-9e23-6142194f720b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:02.825933 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-34608a37-0a5d-4387-9e23-6142194f720b None None] MFA Rules not processed for user `17ff1e357746447a86d47d19daa18cf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:02.827170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-34608a37-0a5d-4387-9e23-6142194f720b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:02.835556 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1340/2682] 10.222.0.75 () {58 vars in 967 bytes} [Fri Sep 23 15:28:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:02.852492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafb34a-3c09-4275-bfd3-6b026ea14436 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:02.861926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafb34a-3c09-4275-bfd3-6b026ea14436 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:02.867827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c15c94-48a7-4c42-844c-d9a4f0ff76c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8614f90cebb4a4eaf7ab5512927fb3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:02.868442 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c15c94-48a7-4c42-844c-d9a4f0ff76c9 service swift] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:02.868866 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c15c94-48a7-4c42-844c-d9a4f0ff76c9 service swift] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:02.869010 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c15c94-48a7-4c42-844c-d9a4f0ff76c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:02.877923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c15c94-48a7-4c42-844c-d9a4f0ff76c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:02.879514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c15c94-48a7-4c42-844c-d9a4f0ff76c9 service swift] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:02.887838 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1343/2683] 10.222.0.75 () {60 vars in 1384 bytes} [Fri Sep 23 15:28:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 15:28:12.817165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=29 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:12.884706 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=12 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:15.942356 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34608a37-0a5d-4387-9e23-6142194f720b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:15.943895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-34608a37-0a5d-4387-9e23-6142194f720b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:16.006030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34608a37-0a5d-4387-9e23-6142194f720b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:16.009574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb0b02-2ebd-49dc-868f-63ef8b842223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:16.010188 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb0b02-2ebd-49dc-868f-63ef8b842223 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:16.010467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb0b02-2ebd-49dc-868f-63ef8b842223 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:16.010732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb0b02-2ebd-49dc-868f-63ef8b842223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:16.012499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-52cb0b02-2ebd-49dc-868f-63ef8b842223 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:16.095311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb0b02-2ebd-49dc-868f-63ef8b842223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:16.096745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb0b02-2ebd-49dc-868f-63ef8b842223 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:16.116337 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1341/2684] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:18.610574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c15c94-48a7-4c42-844c-d9a4f0ff76c9 service swift] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:18.621253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c15c94-48a7-4c42-844c-d9a4f0ff76c9 service swift] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:18.627182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76c279-2aa5-4f52-b2a5-7f7526e28a1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:18.627182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76c279-2aa5-4f52-b2a5-7f7526e28a1f service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:18.627182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76c279-2aa5-4f52-b2a5-7f7526e28a1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:18.627182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76c279-2aa5-4f52-b2a5-7f7526e28a1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:18.636274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76c279-2aa5-4f52-b2a5-7f7526e28a1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:18.637810 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76c279-2aa5-4f52-b2a5-7f7526e28a1f service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:18.648227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1344/2685] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:26.113513 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=61 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:28.611927 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb0b02-2ebd-49dc-868f-63ef8b842223 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:28.624405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb0b02-2ebd-49dc-868f-63ef8b842223 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:28.628567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73078105-ca53-4149-8ecb-5aa11a186970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:28.629196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73078105-ca53-4149-8ecb-5aa11a186970 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:28.629468 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73078105-ca53-4149-8ecb-5aa11a186970 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:28.629733 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73078105-ca53-4149-8ecb-5aa11a186970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:28.631891 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-73078105-ca53-4149-8ecb-5aa11a186970 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:28.643048 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:28.700837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73078105-ca53-4149-8ecb-5aa11a186970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:28.702967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73078105-ca53-4149-8ecb-5aa11a186970 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:28.716564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1342/2686] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:31.958739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76c279-2aa5-4f52-b2a5-7f7526e28a1f service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:31.960976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-ed76c279-2aa5-4f52-b2a5-7f7526e28a1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:31.975278 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76c279-2aa5-4f52-b2a5-7f7526e28a1f service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:31.980949 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adce2767-63a3-4759-af51-fcbb69b58c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:31.981574 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce2767-63a3-4759-af51-fcbb69b58c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:31.981845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce2767-63a3-4759-af51-fcbb69b58c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:31.982111 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adce2767-63a3-4759-af51-fcbb69b58c9c admin admin] PATH_INFO: `/v3/users/4bded8b580f34e08b2890eb3430f4b80` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:31.983286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce2767-63a3-4759-af51-fcbb69b58c9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bded8b580f34e08b2890eb3430f4b80)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:31.985261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adce2767-63a3-4759-af51-fcbb69b58c9c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:32.037630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-adce2767-63a3-4759-af51-fcbb69b58c9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bded8b580f34e08b2890eb3430f4b80'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:32.038390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-adce2767-63a3-4759-af51-fcbb69b58c9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bded8b580f34e08b2890eb3430f4b80'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:32.045682 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-adce2767-63a3-4759-af51-fcbb69b58c9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bded8b580f34e08b2890eb3430f4b80'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:32.061607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-adce2767-63a3-4759-af51-fcbb69b58c9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bded8b580f34e08b2890eb3430f4b80'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:32.062747 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-adce2767-63a3-4759-af51-fcbb69b58c9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bded8b580f34e08b2890eb3430f4b80'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:32.080038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1345/2687] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:31 2022] DELETE /identity/v3/users/4bded8b580f34e08b2890eb3430f4b80 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:32.262361 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73078105-ca53-4149-8ecb-5aa11a186970 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:32.274075 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73078105-ca53-4149-8ecb-5aa11a186970 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:32.337023 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2c29d-4f7e-46e9-9d15-9a906744636a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:32.337655 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2c29d-4f7e-46e9-9d15-9a906744636a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:32.337917 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2c29d-4f7e-46e9-9d15-9a906744636a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:32.338176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2c29d-4f7e-46e9-9d15-9a906744636a admin admin] PATH_INFO: `/v3/projects/d4fd7ca907fa4806904e1c00b21b5447` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:32.339397 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2c29d-4f7e-46e9-9d15-9a906744636a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4fd7ca907fa4806904e1c00b21b5447)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:32.341390 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2c29d-4f7e-46e9-9d15-9a906744636a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:32.391357 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b8d2c29d-4f7e-46e9-9d15-9a906744636a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4fd7ca907fa4806904e1c00b21b5447', 'request_id': 'req-b8d2c29d-4f7e-46e9-9d15-9a906744636a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:32.398071 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b8d2c29d-4f7e-46e9-9d15-9a906744636a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4fd7ca907fa4806904e1c00b21b5447', 'request_id': 'req-b8d2c29d-4f7e-46e9-9d15-9a906744636a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:32.403644 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b8d2c29d-4f7e-46e9-9d15-9a906744636a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4fd7ca907fa4806904e1c00b21b5447', 'request_id': 'req-b8d2c29d-4f7e-46e9-9d15-9a906744636a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:32.421911 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b8d2c29d-4f7e-46e9-9d15-9a906744636a admin admin] The token cache is being invalidate because project d4fd7ca907fa4806904e1c00b21b5447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:32.422348 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b8d2c29d-4f7e-46e9-9d15-9a906744636a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:32.423226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b8d2c29d-4f7e-46e9-9d15-9a906744636a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:32.424818 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1343/2688] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:32 2022] DELETE /identity/v3/projects/d4fd7ca907fa4806904e1c00b21b5447 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:32.431352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaf388-dfd2-4d3f-822f-d15073dc1c00 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:32.431755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaf388-dfd2-4d3f-822f-d15073dc1c00 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:32.432159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaf388-dfd2-4d3f-822f-d15073dc1c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:32.461769 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-bedaf388-dfd2-4d3f-822f-d15073dc1c00 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:32.462901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-bedaf388-dfd2-4d3f-822f-d15073dc1c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:32.525687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1346/2689] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:32.534816 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784207af-0752-4656-bd38-b89102b753ab None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:32.535093 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784207af-0752-4656-bd38-b89102b753ab None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:32.535365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784207af-0752-4656-bd38-b89102b753ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:32.558792 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-784207af-0752-4656-bd38-b89102b753ab None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:32.560111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-784207af-0752-4656-bd38-b89102b753ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:32.568553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1344/2690] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:32.573187 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedaf388-dfd2-4d3f-822f-d15073dc1c00 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:32.583408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedaf388-dfd2-4d3f-822f-d15073dc1c00 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:32.588457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4daa96-738b-403b-b733-938345794b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:32.589114 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4daa96-738b-403b-b733-938345794b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:32.589393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4daa96-738b-403b-b733-938345794b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:32.589658 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4daa96-738b-403b-b733-938345794b63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:32.590339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4daa96-738b-403b-b733-938345794b63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:28:32.590673 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4daa96-738b-403b-b733-938345794b63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:32.592306 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4daa96-738b-403b-b733-938345794b63 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:32.605222 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1347/2691] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:28:32.609851 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784207af-0752-4656-bd38-b89102b753ab None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:32.619461 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784207af-0752-4656-bd38-b89102b753ab None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:32.624682 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a19b7-77f4-4fed-8bbb-b0b3d7bdf177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:32.625286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a19b7-77f4-4fed-8bbb-b0b3d7bdf177 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:32.625558 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a19b7-77f4-4fed-8bbb-b0b3d7bdf177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:32.625881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a19b7-77f4-4fed-8bbb-b0b3d7bdf177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:32.626527 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a19b7-77f4-4fed-8bbb-b0b3d7bdf177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:32.628409 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a19b7-77f4-4fed-8bbb-b0b3d7bdf177 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:32.645175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:32.673079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1345/2692] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:28:32 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:32.677251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4daa96-738b-403b-b733-938345794b63 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:32.687249 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4daa96-738b-403b-b733-938345794b63 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:32.750106 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948814cd-92c6-4e1c-b97d-c7c6dd4ac140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:32.750625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948814cd-92c6-4e1c-b97d-c7c6dd4ac140 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:32.750901 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948814cd-92c6-4e1c-b97d-c7c6dd4ac140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:32.751169 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948814cd-92c6-4e1c-b97d-c7c6dd4ac140 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:32.751814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948814cd-92c6-4e1c-b97d-c7c6dd4ac140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:32.753795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948814cd-92c6-4e1c-b97d-c7c6dd4ac140 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:32.761976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:32.778099 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1348/2693] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:28:32 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:32.782651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a19b7-77f4-4fed-8bbb-b0b3d7bdf177 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:32.793147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a19b7-77f4-4fed-8bbb-b0b3d7bdf177 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:32.797821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a931f2f-ad87-4214-9c9f-d5577f8471f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:32.798418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a931f2f-ad87-4214-9c9f-d5577f8471f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:32.798694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a931f2f-ad87-4214-9c9f-d5577f8471f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:32.798964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a931f2f-ad87-4214-9c9f-d5577f8471f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:32.799610 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a931f2f-ad87-4214-9c9f-d5577f8471f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:32.801255 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a931f2f-ad87-4214-9c9f-d5577f8471f5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:32.814117 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1346/2694] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:32.819009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948814cd-92c6-4e1c-b97d-c7c6dd4ac140 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:32.830361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948814cd-92c6-4e1c-b97d-c7c6dd4ac140 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:32.836986 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00089f0-7444-4a7d-ba54-e9c68eba8108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:32.837583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00089f0-7444-4a7d-ba54-e9c68eba8108 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:32.837854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00089f0-7444-4a7d-ba54-e9c68eba8108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:32.838113 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00089f0-7444-4a7d-ba54-e9c68eba8108 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:32.838754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00089f0-7444-4a7d-ba54-e9c68eba8108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:32.840370 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00089f0-7444-4a7d-ba54-e9c68eba8108 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:32.852417 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1349/2695] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:32.856714 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a931f2f-ad87-4214-9c9f-d5577f8471f5 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:32.867383 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a931f2f-ad87-4214-9c9f-d5577f8471f5 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:32.873414 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091b243-3ed7-4773-b72f-33b7d5c973b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:32.874060 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091b243-3ed7-4773-b72f-33b7d5c973b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:32.874323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091b243-3ed7-4773-b72f-33b7d5c973b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:32.874607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091b243-3ed7-4773-b72f-33b7d5c973b4 admin admin] PATH_INFO: `/v3/projects/c13354446d9941a5bb0565a8fb9b4029/users/09e1eb0a273d40938e52c4c38439de45/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:32.889595 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091b243-3ed7-4773-b72f-33b7d5c973b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c13354446d9941a5bb0565a8fb9b4029, user_id=09e1eb0a273d40938e52c4c38439de45, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:32.892184 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091b243-3ed7-4773-b72f-33b7d5c973b4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:32.909712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1347/2696] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:32 2022] PUT /identity/v3/projects/c13354446d9941a5bb0565a8fb9b4029/users/09e1eb0a273d40938e52c4c38439de45/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:32.924495 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c7a9-b82f-4d3e-8c6a-e6e6f7963cf5 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:32.924849 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c7a9-b82f-4d3e-8c6a-e6e6f7963cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:32.925119 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c7a9-b82f-4d3e-8c6a-e6e6f7963cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:32.957235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-6ca1c7a9-b82f-4d3e-8c6a-e6e6f7963cf5 None None] MFA Rules not processed for user `09e1eb0a273d40938e52c4c38439de45`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:32.958480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6ca1c7a9-b82f-4d3e-8c6a-e6e6f7963cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:33.026918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1350/2697] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:33.038244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091b243-3ed7-4773-b72f-33b7d5c973b4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:33.040220 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9091b243-3ed7-4773-b72f-33b7d5c973b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:33.101297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091b243-3ed7-4773-b72f-33b7d5c973b4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:33.105555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a5fad-46d6-45be-b77b-d458ec267ed7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:33.106160 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a5fad-46d6-45be-b77b-d458ec267ed7 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:33.106427 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a5fad-46d6-45be-b77b-d458ec267ed7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:33.106719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a5fad-46d6-45be-b77b-d458ec267ed7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:33.116246 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a5fad-46d6-45be-b77b-d458ec267ed7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:33.117685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a5fad-46d6-45be-b77b-d458ec267ed7 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:33.127424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1348/2698] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:41.722403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1c7a9-b82f-4d3e-8c6a-e6e6f7963cf5 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:41.787265 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1c7a9-b82f-4d3e-8c6a-e6e6f7963cf5 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:41.795797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:41.796850 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:41.797330 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:41.797655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] PATH_INFO: `/v3/users/09e1eb0a273d40938e52c4c38439de45` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:41.799508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e1eb0a273d40938e52c4c38439de45)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:41.802715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:41.857588 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e1eb0a273d40938e52c4c38439de45'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:41.858217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e1eb0a273d40938e52c4c38439de45'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:41.864983 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e1eb0a273d40938e52c4c38439de45'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:41.876625 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:41.879066 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e1eb0a273d40938e52c4c38439de45'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:41.879827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e1eb0a273d40938e52c4c38439de45'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:41.895945 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1351/2699] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:41 2022] DELETE /identity/v3/users/09e1eb0a273d40938e52c4c38439de45 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:41.901040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0edd91-4950-44a7-b9a0-5742ea671e8c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:41.901483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0edd91-4950-44a7-b9a0-5742ea671e8c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:41.901591 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0edd91-4950-44a7-b9a0-5742ea671e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:41.921058 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5a0edd91-4950-44a7-b9a0-5742ea671e8c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:41.922248 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5a0edd91-4950-44a7-b9a0-5742ea671e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:41.984463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1349/2700] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:41.994742 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:41.996238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:42.055560 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84253c02-c2b3-4d84-b97c-4b6568e84cc3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:42.059444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1d4c3-4af8-45f4-a423-38cb31090dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:42.060087 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1d4c3-4af8-45f4-a423-38cb31090dbf service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:42.060364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1d4c3-4af8-45f4-a423-38cb31090dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:42.060651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1d4c3-4af8-45f4-a423-38cb31090dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:42.070228 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1d4c3-4af8-45f4-a423-38cb31090dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:42.071875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1d4c3-4af8-45f4-a423-38cb31090dbf service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:42.081454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1352/2701] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:42.371408 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0edd91-4950-44a7-b9a0-5742ea671e8c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:42.373676 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5a0edd91-4950-44a7-b9a0-5742ea671e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:42.387710 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0edd91-4950-44a7-b9a0-5742ea671e8c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:42.392606 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:42.393249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:42.393518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:42.393784 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9 admin admin] PATH_INFO: `/v3/projects/c13354446d9941a5bb0565a8fb9b4029` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:42.394943 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c13354446d9941a5bb0565a8fb9b4029)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:42.396886 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:42.446482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c13354446d9941a5bb0565a8fb9b4029', 'request_id': 'req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:42.447520 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:42.454391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c13354446d9941a5bb0565a8fb9b4029', 'request_id': 'req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:42.461435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c13354446d9941a5bb0565a8fb9b4029', 'request_id': 'req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:42.476735 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9 admin admin] The token cache is being invalidate because project c13354446d9941a5bb0565a8fb9b4029 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:42.476903 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:42.477457 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-ce9dc873-213c-4921-9ffe-bc9f0fc452f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:42.478960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1350/2702] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:42 2022] DELETE /identity/v3/projects/c13354446d9941a5bb0565a8fb9b4029 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:42.484032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27643660-9d2a-4144-9813-ee42ea47ed46 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:42.484443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27643660-9d2a-4144-9813-ee42ea47ed46 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:42.484817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27643660-9d2a-4144-9813-ee42ea47ed46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:42.505618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-27643660-9d2a-4144-9813-ee42ea47ed46 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:42.506830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-27643660-9d2a-4144-9813-ee42ea47ed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:42.574426 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1353/2703] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:42.582590 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb552f-9a23-4f13-9953-ef0d5fc41b3b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:42.582868 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb552f-9a23-4f13-9953-ef0d5fc41b3b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:42.583134 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb552f-9a23-4f13-9953-ef0d5fc41b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:42.605140 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cfeb552f-9a23-4f13-9953-ef0d5fc41b3b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:42.606346 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cfeb552f-9a23-4f13-9953-ef0d5fc41b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:42.615805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1351/2704] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:42.620168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27643660-9d2a-4144-9813-ee42ea47ed46 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:42.630817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27643660-9d2a-4144-9813-ee42ea47ed46 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:42.637093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221f91d-6868-456b-bee7-fecffe5dcdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:42.637678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221f91d-6868-456b-bee7-fecffe5dcdee admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:42.637952 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221f91d-6868-456b-bee7-fecffe5dcdee admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:42.638205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221f91d-6868-456b-bee7-fecffe5dcdee admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:42.638862 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221f91d-6868-456b-bee7-fecffe5dcdee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:28:42.639159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221f91d-6868-456b-bee7-fecffe5dcdee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:42.640979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221f91d-6868-456b-bee7-fecffe5dcdee admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:42.654101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1354/2705] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:28:42.659954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb552f-9a23-4f13-9953-ef0d5fc41b3b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:42.669645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb552f-9a23-4f13-9953-ef0d5fc41b3b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:42.675257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173b8a8-8ba8-4b29-8fd8-e00e71390faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:42.675860 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173b8a8-8ba8-4b29-8fd8-e00e71390faf admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:42.676136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173b8a8-8ba8-4b29-8fd8-e00e71390faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:42.676403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173b8a8-8ba8-4b29-8fd8-e00e71390faf admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:42.677078 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173b8a8-8ba8-4b29-8fd8-e00e71390faf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:42.678967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7173b8a8-8ba8-4b29-8fd8-e00e71390faf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:42.709547 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1352/2706] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:28:42 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:42.713904 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221f91d-6868-456b-bee7-fecffe5dcdee admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:42.724135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221f91d-6868-456b-bee7-fecffe5dcdee admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:42.788117 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62477f-f57a-4df2-ab0f-5ed23d73a88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:42.789154 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62477f-f57a-4df2-ab0f-5ed23d73a88f admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:42.789423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62477f-f57a-4df2-ab0f-5ed23d73a88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:42.789681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62477f-f57a-4df2-ab0f-5ed23d73a88f admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:42.790307 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62477f-f57a-4df2-ab0f-5ed23d73a88f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:42.792217 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62477f-f57a-4df2-ab0f-5ed23d73a88f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:42.816715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1355/2707] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:28:42 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:42.821536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173b8a8-8ba8-4b29-8fd8-e00e71390faf admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:42.832449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173b8a8-8ba8-4b29-8fd8-e00e71390faf admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:42.838342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ec371-f97a-4dd3-a38f-16803c17b7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:42.838936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ec371-f97a-4dd3-a38f-16803c17b7ab admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:42.839168 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ec371-f97a-4dd3-a38f-16803c17b7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:42.839464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ec371-f97a-4dd3-a38f-16803c17b7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:42.840101 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ec371-f97a-4dd3-a38f-16803c17b7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:42.841734 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ec371-f97a-4dd3-a38f-16803c17b7ab admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:42.852297 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1353/2708] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:42.856412 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62477f-f57a-4df2-ab0f-5ed23d73a88f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:42.866701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62477f-f57a-4df2-ab0f-5ed23d73a88f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:42.873414 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b823a-40c2-4616-8222-194a3fb82869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:42.874029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b823a-40c2-4616-8222-194a3fb82869 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:42.874312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b823a-40c2-4616-8222-194a3fb82869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:42.874570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b823a-40c2-4616-8222-194a3fb82869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:42.875206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b823a-40c2-4616-8222-194a3fb82869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:42.876827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b823a-40c2-4616-8222-194a3fb82869 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:42.878062 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:42.893567 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1356/2709] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:42.894586 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ec371-f97a-4dd3-a38f-16803c17b7ab admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:42.904833 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ec371-f97a-4dd3-a38f-16803c17b7ab admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:42.909752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11c78a-438a-4f1d-a5fa-604e5f6d0e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:42.910394 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11c78a-438a-4f1d-a5fa-604e5f6d0e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:42.910661 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11c78a-438a-4f1d-a5fa-604e5f6d0e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:42.910919 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11c78a-438a-4f1d-a5fa-604e5f6d0e3f admin admin] PATH_INFO: `/v3/projects/87f978a6d63843e2937a36adbb582272/users/5031ea6eb93a43558688913c0e7b1de7/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:42.926436 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11c78a-438a-4f1d-a5fa-604e5f6d0e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=87f978a6d63843e2937a36adbb582272, user_id=5031ea6eb93a43558688913c0e7b1de7, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:42.929090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11c78a-438a-4f1d-a5fa-604e5f6d0e3f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:42.945901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1354/2710] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:42 2022] PUT /identity/v3/projects/87f978a6d63843e2937a36adbb582272/users/5031ea6eb93a43558688913c0e7b1de7/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:42.957894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5983c73-98c6-4691-a4dd-fd7e6504a61d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:42.958171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5983c73-98c6-4691-a4dd-fd7e6504a61d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:42.958439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5983c73-98c6-4691-a4dd-fd7e6504a61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:42.986368 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f5983c73-98c6-4691-a4dd-fd7e6504a61d None None] MFA Rules not processed for user `5031ea6eb93a43558688913c0e7b1de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:42.987637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f5983c73-98c6-4691-a4dd-fd7e6504a61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:43.055458 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1357/2711] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:43.060231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11c78a-438a-4f1d-a5fa-604e5f6d0e3f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:43.137499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11c78a-438a-4f1d-a5fa-604e5f6d0e3f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:43.144625 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90e656-b3d1-46c9-a79f-0de232294411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:43.145240 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90e656-b3d1-46c9-a79f-0de232294411 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:43.145517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90e656-b3d1-46c9-a79f-0de232294411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:43.145775 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90e656-b3d1-46c9-a79f-0de232294411 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:43.146405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90e656-b3d1-46c9-a79f-0de232294411 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:43.148281 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90e656-b3d1-46c9-a79f-0de232294411 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:43.178885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1355/2712] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:28:43 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:43.183196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5983c73-98c6-4691-a4dd-fd7e6504a61d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:43.194437 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5983c73-98c6-4691-a4dd-fd7e6504a61d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:43.254573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d11eae-a11d-4b47-878a-3ed4d5f43385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:43.255148 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d11eae-a11d-4b47-878a-3ed4d5f43385 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:43.255454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d11eae-a11d-4b47-878a-3ed4d5f43385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:43.255718 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d11eae-a11d-4b47-878a-3ed4d5f43385 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:43.256363 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d11eae-a11d-4b47-878a-3ed4d5f43385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:43.258371 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d11eae-a11d-4b47-878a-3ed4d5f43385 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:43.282613 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1358/2713] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:28:43 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:43.286789 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90e656-b3d1-46c9-a79f-0de232294411 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:43.297128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90e656-b3d1-46c9-a79f-0de232294411 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:43.303736 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132716e-16a9-48a6-918d-0e701883d559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:43.304654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132716e-16a9-48a6-918d-0e701883d559 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:43.305019 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132716e-16a9-48a6-918d-0e701883d559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:43.305285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132716e-16a9-48a6-918d-0e701883d559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:43.306165 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132716e-16a9-48a6-918d-0e701883d559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:43.307758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132716e-16a9-48a6-918d-0e701883d559 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:43.318796 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1356/2714] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:43.323420 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d11eae-a11d-4b47-878a-3ed4d5f43385 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:43.333922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d11eae-a11d-4b47-878a-3ed4d5f43385 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:43.339095 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d82699-da28-4583-a9dc-5bbee2dc4be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:43.339739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d82699-da28-4583-a9dc-5bbee2dc4be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:43.340042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d82699-da28-4583-a9dc-5bbee2dc4be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:43.340309 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d82699-da28-4583-a9dc-5bbee2dc4be9 admin admin] PATH_INFO: `/v3/projects/da44b71820bb4713a72d5aafdad43b67/users/33be9f2637a740d3989d3826f6870905/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:43.355493 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d82699-da28-4583-a9dc-5bbee2dc4be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da44b71820bb4713a72d5aafdad43b67, user_id=33be9f2637a740d3989d3826f6870905, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:43.358182 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d82699-da28-4583-a9dc-5bbee2dc4be9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:43.374037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1359/2715] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:43 2022] PUT /identity/v3/projects/da44b71820bb4713a72d5aafdad43b67/users/33be9f2637a740d3989d3826f6870905/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:43.378230 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132716e-16a9-48a6-918d-0e701883d559 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:43.388673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132716e-16a9-48a6-918d-0e701883d559 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:43.436105 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:43.452166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963bc2f-9255-4fc9-bf00-fe1c23496ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:43.452839 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963bc2f-9255-4fc9-bf00-fe1c23496ddc admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:43.453126 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963bc2f-9255-4fc9-bf00-fe1c23496ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:43.453384 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963bc2f-9255-4fc9-bf00-fe1c23496ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:43.454029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963bc2f-9255-4fc9-bf00-fe1c23496ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:43.455619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963bc2f-9255-4fc9-bf00-fe1c23496ddc admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:43.465823 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1357/2716] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:28:43.470341 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d82699-da28-4583-a9dc-5bbee2dc4be9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:43.481493 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d82699-da28-4583-a9dc-5bbee2dc4be9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:43.487500 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb7e62-beac-4e08-ba59-e81d1a006edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:43.488157 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb7e62-beac-4e08-ba59-e81d1a006edf admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:43.488436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb7e62-beac-4e08-ba59-e81d1a006edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:43.488696 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb7e62-beac-4e08-ba59-e81d1a006edf admin admin] PATH_INFO: `/v3/projects/da44b71820bb4713a72d5aafdad43b67/users/33be9f2637a740d3989d3826f6870905/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:43.490873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb7e62-beac-4e08-ba59-e81d1a006edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=da44b71820bb4713a72d5aafdad43b67, user_id=33be9f2637a740d3989d3826f6870905, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:43.493712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb7e62-beac-4e08-ba59-e81d1a006edf admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:43.504756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1360/2717] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:43 2022] PUT /identity/v3/projects/da44b71820bb4713a72d5aafdad43b67/users/33be9f2637a740d3989d3826f6870905/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:43.515439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc3e9b-a9b6-41e2-81a7-9c23b85842b9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:43.515721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc3e9b-a9b6-41e2-81a7-9c23b85842b9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:43.515992 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc3e9b-a9b6-41e2-81a7-9c23b85842b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:43.545745 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-7fcc3e9b-a9b6-41e2-81a7-9c23b85842b9 None None] MFA Rules not processed for user `33be9f2637a740d3989d3826f6870905`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:43.546967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7fcc3e9b-a9b6-41e2-81a7-9c23b85842b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:43.617668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1358/2718] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:43.630197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb7e62-beac-4e08-ba59-e81d1a006edf admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:43.633367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1afb7e62-beac-4e08-ba59-e81d1a006edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:43.677723 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:43.696875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb7e62-beac-4e08-ba59-e81d1a006edf admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:43.700820 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f4642-091f-4fca-be95-02922603dd90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:43.701423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f4642-091f-4fca-be95-02922603dd90 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:43.701698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f4642-091f-4fca-be95-02922603dd90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:43.701966 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f4642-091f-4fca-be95-02922603dd90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:43.711238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f4642-091f-4fca-be95-02922603dd90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:43.712701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f4642-091f-4fca-be95-02922603dd90 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:43.722000 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1361/2719] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:43.771650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc3e9b-a9b6-41e2-81a7-9c23b85842b9 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:43.773345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-7fcc3e9b-a9b6-41e2-81a7-9c23b85842b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:43.786580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc3e9b-a9b6-41e2-81a7-9c23b85842b9 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:43.790768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f72fa-e423-4141-8cfc-88d3a5c74ed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:43.791374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f72fa-e423-4141-8cfc-88d3a5c74ed1 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:43.791659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f72fa-e423-4141-8cfc-88d3a5c74ed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:43.791910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f72fa-e423-4141-8cfc-88d3a5c74ed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:43.849331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f72fa-e423-4141-8cfc-88d3a5c74ed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:43.850761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f72fa-e423-4141-8cfc-88d3a5c74ed1 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:43.861452 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1359/2720] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:47.273054 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f4642-091f-4fca-be95-02922603dd90 service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:47.337418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f4642-091f-4fca-be95-02922603dd90 service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:47.343038 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:47.343652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:47.343924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:47.344190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] PATH_INFO: `/v3/users/5031ea6eb93a43558688913c0e7b1de7` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:47.345396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5031ea6eb93a43558688913c0e7b1de7)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:47.347320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:47.395752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5031ea6eb93a43558688913c0e7b1de7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:47.396382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5031ea6eb93a43558688913c0e7b1de7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:47.403384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5031ea6eb93a43558688913c0e7b1de7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:47.415321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5031ea6eb93a43558688913c0e7b1de7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:47.416007 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5031ea6eb93a43558688913c0e7b1de7'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:47.430263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1362/2721] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:47 2022] DELETE /identity/v3/users/5031ea6eb93a43558688913c0e7b1de7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:47.435140 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc909b-8164-4cbe-9776-d379314f594a None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:47.435654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc909b-8164-4cbe-9776-d379314f594a None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:47.435654 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc909b-8164-4cbe-9776-d379314f594a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:47.455529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4abc909b-8164-4cbe-9776-d379314f594a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:47.456658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4abc909b-8164-4cbe-9776-d379314f594a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:47.515378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1360/2722] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:47.525340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:47.527097 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:47.586104 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb137a18-8662-4d52-bf6c-3970b9949c08 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:47.590257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74a38a-e783-4517-8981-4609c5608c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:47.590860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74a38a-e783-4517-8981-4609c5608c31 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:47.591132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74a38a-e783-4517-8981-4609c5608c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:47.591396 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74a38a-e783-4517-8981-4609c5608c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:47.604453 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74a38a-e783-4517-8981-4609c5608c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:47.607243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74a38a-e783-4517-8981-4609c5608c31 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:47.621766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1363/2723] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:47.902888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc909b-8164-4cbe-9776-d379314f594a None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:47.904381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4abc909b-8164-4cbe-9776-d379314f594a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:47.916290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc909b-8164-4cbe-9776-d379314f594a None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:47.921895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:47.922564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:47.922822 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:47.923095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] PATH_INFO: `/v3/projects/87f978a6d63843e2937a36adbb582272` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:47.924261 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87f978a6d63843e2937a36adbb582272)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:47.926398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:47.929047 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=100 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:47.978897 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87f978a6d63843e2937a36adbb582272', 'request_id': 'req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:47.985829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87f978a6d63843e2937a36adbb582272', 'request_id': 'req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:47.991843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87f978a6d63843e2937a36adbb582272', 'request_id': 'req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:48.007776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] The token cache is being invalidate because project 87f978a6d63843e2937a36adbb582272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:48.008189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:48.009095 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:48.011306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1361/2724] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:47 2022] DELETE /identity/v3/projects/87f978a6d63843e2937a36adbb582272 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:28:48.015538 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74a38a-e783-4517-8981-4609c5608c31 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:48.020386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:48.026492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74a38a-e783-4517-8981-4609c5608c31 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:48.090351 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f9280-abff-479c-b34d-fe67c5796db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:48.091010 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f9280-abff-479c-b34d-fe67c5796db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:48.091280 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f9280-abff-479c-b34d-fe67c5796db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:48.091543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f9280-abff-479c-b34d-fe67c5796db0 admin admin] PATH_INFO: `/v3/users/33be9f2637a740d3989d3826f6870905` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:48.092938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f9280-abff-479c-b34d-fe67c5796db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33be9f2637a740d3989d3826f6870905)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:48.094895 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f9280-abff-479c-b34d-fe67c5796db0 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:48.151950 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fe9f9280-abff-479c-b34d-fe67c5796db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33be9f2637a740d3989d3826f6870905'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:48.152661 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fe9f9280-abff-479c-b34d-fe67c5796db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33be9f2637a740d3989d3826f6870905'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:48.159547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fe9f9280-abff-479c-b34d-fe67c5796db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33be9f2637a740d3989d3826f6870905'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:48.171850 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fe9f9280-abff-479c-b34d-fe67c5796db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33be9f2637a740d3989d3826f6870905'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:48.172498 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-fe9f9280-abff-479c-b34d-fe67c5796db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33be9f2637a740d3989d3826f6870905'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:48.189207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1364/2725] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:48 2022] DELETE /identity/v3/users/33be9f2637a740d3989d3826f6870905 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:28:48.482908 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:48.484543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:48.499167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22dc11-ed78-4d93-9ee1-23e6bd4e7e46 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:48.560226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2816b05-4aa3-4524-9be8-74a5c72971f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:48.560893 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2816b05-4aa3-4524-9be8-74a5c72971f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:48.561194 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2816b05-4aa3-4524-9be8-74a5c72971f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:48.561463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2816b05-4aa3-4524-9be8-74a5c72971f8 admin admin] PATH_INFO: `/v3/projects/da44b71820bb4713a72d5aafdad43b67` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:48.562603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2816b05-4aa3-4524-9be8-74a5c72971f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da44b71820bb4713a72d5aafdad43b67)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:48.564490 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2816b05-4aa3-4524-9be8-74a5c72971f8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:48.613419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d2816b05-4aa3-4524-9be8-74a5c72971f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da44b71820bb4713a72d5aafdad43b67', 'request_id': 'req-d2816b05-4aa3-4524-9be8-74a5c72971f8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:48.619929 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d2816b05-4aa3-4524-9be8-74a5c72971f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da44b71820bb4713a72d5aafdad43b67', 'request_id': 'req-d2816b05-4aa3-4524-9be8-74a5c72971f8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:48.626142 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d2816b05-4aa3-4524-9be8-74a5c72971f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da44b71820bb4713a72d5aafdad43b67', 'request_id': 'req-d2816b05-4aa3-4524-9be8-74a5c72971f8'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:48.640605 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d2816b05-4aa3-4524-9be8-74a5c72971f8 admin admin] The token cache is being invalidate because project da44b71820bb4713a72d5aafdad43b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:48.640904 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d2816b05-4aa3-4524-9be8-74a5c72971f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:48.641574 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d2816b05-4aa3-4524-9be8-74a5c72971f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:48.643139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1362/2726] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:48 2022] DELETE /identity/v3/projects/da44b71820bb4713a72d5aafdad43b67 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:48.648390 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d10c5-a851-485f-ae08-ef525d39dc0c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:48.648951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d10c5-a851-485f-ae08-ef525d39dc0c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:48.648951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d10c5-a851-485f-ae08-ef525d39dc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:48.670184 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f06d10c5-a851-485f-ae08-ef525d39dc0c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:48.671378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f06d10c5-a851-485f-ae08-ef525d39dc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:48.734731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1365/2727] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:48.742950 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f0f16-7d7e-4374-87b2-199da8ff01cf None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:48.743230 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f0f16-7d7e-4374-87b2-199da8ff01cf None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:48.743503 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f0f16-7d7e-4374-87b2-199da8ff01cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:48.764622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-841f0f16-7d7e-4374-87b2-199da8ff01cf None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:48.766027 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-841f0f16-7d7e-4374-87b2-199da8ff01cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:48.774387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1363/2728] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:48.778900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d10c5-a851-485f-ae08-ef525d39dc0c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:48.789227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d10c5-a851-485f-ae08-ef525d39dc0c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:48.795205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad74a77-7f83-4657-94ff-3a5fc17c9633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:48.795797 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad74a77-7f83-4657-94ff-3a5fc17c9633 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:48.796061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad74a77-7f83-4657-94ff-3a5fc17c9633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:48.796320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad74a77-7f83-4657-94ff-3a5fc17c9633 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:48.796999 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad74a77-7f83-4657-94ff-3a5fc17c9633 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:28:48.797292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad74a77-7f83-4657-94ff-3a5fc17c9633 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:48.798894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad74a77-7f83-4657-94ff-3a5fc17c9633 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:48.811350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1366/2729] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:28:48.815768 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f0f16-7d7e-4374-87b2-199da8ff01cf None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:48.825686 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f0f16-7d7e-4374-87b2-199da8ff01cf None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:48.830729 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee00627-2322-4d5c-af01-f6178a664a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:48.831311 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee00627-2322-4d5c-af01-f6178a664a6e admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:48.831583 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee00627-2322-4d5c-af01-f6178a664a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:48.831845 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee00627-2322-4d5c-af01-f6178a664a6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:48.832473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee00627-2322-4d5c-af01-f6178a664a6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:48.834375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee00627-2322-4d5c-af01-f6178a664a6e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:48.850566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:48.866902 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1364/2730] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:28:48 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:48.871557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad74a77-7f83-4657-94ff-3a5fc17c9633 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:48.882714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad74a77-7f83-4657-94ff-3a5fc17c9633 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:48.888645 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:48.942621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2ccd6-36be-41bd-a296-62b7063c551c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:48.943207 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2ccd6-36be-41bd-a296-62b7063c551c admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:48.943487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2ccd6-36be-41bd-a296-62b7063c551c admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:48.943751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2ccd6-36be-41bd-a296-62b7063c551c admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:48.944375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2ccd6-36be-41bd-a296-62b7063c551c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:48.946322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2ccd6-36be-41bd-a296-62b7063c551c admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:48.969615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1367/2731] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:28:48 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:48.974172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee00627-2322-4d5c-af01-f6178a664a6e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:48.984925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee00627-2322-4d5c-af01-f6178a664a6e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:48.989752 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7737328-3f22-4ef5-9d88-68384a73edf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:48.990351 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7737328-3f22-4ef5-9d88-68384a73edf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:48.990623 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7737328-3f22-4ef5-9d88-68384a73edf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:48.990892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7737328-3f22-4ef5-9d88-68384a73edf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:48.991563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7737328-3f22-4ef5-9d88-68384a73edf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:48.993222 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7737328-3f22-4ef5-9d88-68384a73edf4 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:49.004000 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1365/2732] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:49.008061 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2ccd6-36be-41bd-a296-62b7063c551c admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:49.018548 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2ccd6-36be-41bd-a296-62b7063c551c admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:49.024179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fd264-a34c-404c-859c-e5b6877a7b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:49.024813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fd264-a34c-404c-859c-e5b6877a7b10 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:49.025113 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fd264-a34c-404c-859c-e5b6877a7b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:49.025383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fd264-a34c-404c-859c-e5b6877a7b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:49.026039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fd264-a34c-404c-859c-e5b6877a7b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:49.027771 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fd264-a34c-404c-859c-e5b6877a7b10 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:49.038794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1368/2733] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:49.043162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7737328-3f22-4ef5-9d88-68384a73edf4 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:49.053687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7737328-3f22-4ef5-9d88-68384a73edf4 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:49.059473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b69d56-da19-412f-a1e9-a4216d01677a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:49.060061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b69d56-da19-412f-a1e9-a4216d01677a admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:49.060367 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b69d56-da19-412f-a1e9-a4216d01677a admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:49.060623 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b69d56-da19-412f-a1e9-a4216d01677a admin admin] PATH_INFO: `/v3/projects/cd1f835336b64fcc9c850f69c4d8b711/users/a7bc09ef56b8419eae7419b0d4d4fa13/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:49.075906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b69d56-da19-412f-a1e9-a4216d01677a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd1f835336b64fcc9c850f69c4d8b711, user_id=a7bc09ef56b8419eae7419b0d4d4fa13, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:49.078566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b69d56-da19-412f-a1e9-a4216d01677a admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:49.097322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1366/2734] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:49 2022] PUT /identity/v3/projects/cd1f835336b64fcc9c850f69c4d8b711/users/a7bc09ef56b8419eae7419b0d4d4fa13/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:49.109142 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6c7c7-dfc8-4e2c-939d-a2b0bb0ef76a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:49.109426 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6c7c7-dfc8-4e2c-939d-a2b0bb0ef76a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:49.109726 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6c7c7-dfc8-4e2c-939d-a2b0bb0ef76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:49.140830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0dd6c7c7-dfc8-4e2c-939d-a2b0bb0ef76a None None] MFA Rules not processed for user `a7bc09ef56b8419eae7419b0d4d4fa13`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:49.142089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0dd6c7c7-dfc8-4e2c-939d-a2b0bb0ef76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:49.209360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1369/2735] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:49.214258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b69d56-da19-412f-a1e9-a4216d01677a admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:49.282569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b69d56-da19-412f-a1e9-a4216d01677a admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:49.287589 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da1910-4422-44dd-a31d-71bd3c960a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:49.288194 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da1910-4422-44dd-a31d-71bd3c960a60 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:49.288489 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da1910-4422-44dd-a31d-71bd3c960a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:49.288786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da1910-4422-44dd-a31d-71bd3c960a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:49.289391 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da1910-4422-44dd-a31d-71bd3c960a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:49.291273 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da1910-4422-44dd-a31d-71bd3c960a60 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:49.321912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1367/2736] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:28:49 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:49.326274 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6c7c7-dfc8-4e2c-939d-a2b0bb0ef76a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:49.337253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd6c7c7-dfc8-4e2c-939d-a2b0bb0ef76a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:49.400895 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fcccb-3c77-4556-933d-201d5472d9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:49.401487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fcccb-3c77-4556-933d-201d5472d9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:49.401773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fcccb-3c77-4556-933d-201d5472d9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:49.402042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fcccb-3c77-4556-933d-201d5472d9d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:49.402662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fcccb-3c77-4556-933d-201d5472d9d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:49.404605 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fcccb-3c77-4556-933d-201d5472d9d9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:49.427988 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1370/2737] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:28:49 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:49.432036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da1910-4422-44dd-a31d-71bd3c960a60 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:49.442466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da1910-4422-44dd-a31d-71bd3c960a60 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:49.447321 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe810d-ddf2-4c4c-8853-4a1877313e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:49.447915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe810d-ddf2-4c4c-8853-4a1877313e3d admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:49.448213 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe810d-ddf2-4c4c-8853-4a1877313e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:49.449053 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe810d-ddf2-4c4c-8853-4a1877313e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:49.449707 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe810d-ddf2-4c4c-8853-4a1877313e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:49.451284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbe810d-ddf2-4c4c-8853-4a1877313e3d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:49.461739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1368/2738] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:49.466059 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fcccb-3c77-4556-933d-201d5472d9d9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:49.476234 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fcccb-3c77-4556-933d-201d5472d9d9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:49.481902 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ba30b-054c-42cd-b507-e43fd883e70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:49.482600 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ba30b-054c-42cd-b507-e43fd883e70e admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:49.482855 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ba30b-054c-42cd-b507-e43fd883e70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:49.483130 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ba30b-054c-42cd-b507-e43fd883e70e admin admin] PATH_INFO: `/v3/projects/c997f98799164e8f86d3eb7a7c6e6a16/users/87f3373fbdf246b8bd7d6b319c8d3e3a/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:49.485524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:49.500553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ba30b-054c-42cd-b507-e43fd883e70e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c997f98799164e8f86d3eb7a7c6e6a16, user_id=87f3373fbdf246b8bd7d6b319c8d3e3a, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:49.503259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94ba30b-054c-42cd-b507-e43fd883e70e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:49.518427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1371/2739] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:49 2022] PUT /identity/v3/projects/c997f98799164e8f86d3eb7a7c6e6a16/users/87f3373fbdf246b8bd7d6b319c8d3e3a/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:49.522468 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe810d-ddf2-4c4c-8853-4a1877313e3d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:49.533127 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe810d-ddf2-4c4c-8853-4a1877313e3d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:49.600151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086066aa-73ee-41e8-8777-389edbe87861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:49.600934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086066aa-73ee-41e8-8777-389edbe87861 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:49.601267 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086066aa-73ee-41e8-8777-389edbe87861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:49.601572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086066aa-73ee-41e8-8777-389edbe87861 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:49.602481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086066aa-73ee-41e8-8777-389edbe87861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:49.604988 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086066aa-73ee-41e8-8777-389edbe87861 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:49.614783 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1369/2740] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:49.619241 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ba30b-054c-42cd-b507-e43fd883e70e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:49.630347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ba30b-054c-42cd-b507-e43fd883e70e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:49.635577 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f6f61-c738-4426-a07e-8fbe0684d73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:49.636335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f6f61-c738-4426-a07e-8fbe0684d73e admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:49.636762 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f6f61-c738-4426-a07e-8fbe0684d73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:49.637031 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f6f61-c738-4426-a07e-8fbe0684d73e admin admin] PATH_INFO: `/v3/projects/c997f98799164e8f86d3eb7a7c6e6a16/users/87f3373fbdf246b8bd7d6b319c8d3e3a/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:49.639082 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f6f61-c738-4426-a07e-8fbe0684d73e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c997f98799164e8f86d3eb7a7c6e6a16, user_id=87f3373fbdf246b8bd7d6b319c8d3e3a, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:49.641729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08f6f61-c738-4426-a07e-8fbe0684d73e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:49.652573 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1372/2741] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:49 2022] PUT /identity/v3/projects/c997f98799164e8f86d3eb7a7c6e6a16/users/87f3373fbdf246b8bd7d6b319c8d3e3a/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:49.663557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195523b-56aa-45a0-8ee3-b76892c78f37 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:49.663838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195523b-56aa-45a0-8ee3-b76892c78f37 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:49.664106 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195523b-56aa-45a0-8ee3-b76892c78f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:49.699164 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c195523b-56aa-45a0-8ee3-b76892c78f37 None None] MFA Rules not processed for user `87f3373fbdf246b8bd7d6b319c8d3e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:49.700396 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c195523b-56aa-45a0-8ee3-b76892c78f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:49.718429 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:49.779565 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1370/2742] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:49.794782 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f6f61-c738-4426-a07e-8fbe0684d73e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:49.797845 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a08f6f61-c738-4426-a07e-8fbe0684d73e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:49.863404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f6f61-c738-4426-a07e-8fbe0684d73e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:49.867895 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e05510-6f1e-45e4-8a76-b296708d4e7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:49.868725 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e05510-6f1e-45e4-8a76-b296708d4e7a service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:49.868902 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e05510-6f1e-45e4-8a76-b296708d4e7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:49.869096 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e05510-6f1e-45e4-8a76-b296708d4e7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:49.879324 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e05510-6f1e-45e4-8a76-b296708d4e7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:49.880213 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e05510-6f1e-45e4-8a76-b296708d4e7a service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:49.889997 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1373/2743] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:49.919161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195523b-56aa-45a0-8ee3-b76892c78f37 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:49.983629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195523b-56aa-45a0-8ee3-b76892c78f37 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:49.988845 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:49.989466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:49.989737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:49.989987 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] PATH_INFO: `/v3/users/a7bc09ef56b8419eae7419b0d4d4fa13` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:49.991157 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7bc09ef56b8419eae7419b0d4d4fa13)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:49.993110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:50.051424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7bc09ef56b8419eae7419b0d4d4fa13'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.058953 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7bc09ef56b8419eae7419b0d4d4fa13'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.071399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7bc09ef56b8419eae7419b0d4d4fa13'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.080335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7bc09ef56b8419eae7419b0d4d4fa13'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.080998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7bc09ef56b8419eae7419b0d4d4fa13'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.087525 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1371/2744] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:49 2022] DELETE /identity/v3/users/a7bc09ef56b8419eae7419b0d4d4fa13 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:50.092254 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182502c0-8250-47ab-b833-6e4826573eb4 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:50.092553 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182502c0-8250-47ab-b833-6e4826573eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:50.092875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182502c0-8250-47ab-b833-6e4826573eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:50.113174 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-182502c0-8250-47ab-b833-6e4826573eb4 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:50.114405 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-182502c0-8250-47ab-b833-6e4826573eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:50.179905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1374/2745] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:50.189925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:50.191450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:50.229190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:50.254843 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47856b75-6568-4494-812f-1b4a6c8e4d7c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:50.258581 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497d9af-afa2-4572-9492-922874986690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:50.259179 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497d9af-afa2-4572-9492-922874986690 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:50.259451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497d9af-afa2-4572-9492-922874986690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:50.259719 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a497d9af-afa2-4572-9492-922874986690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:50.268857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497d9af-afa2-4572-9492-922874986690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:50.270272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a497d9af-afa2-4572-9492-922874986690 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:50.279477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1372/2746] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:50.560112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182502c0-8250-47ab-b833-6e4826573eb4 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:50.561598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-182502c0-8250-47ab-b833-6e4826573eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:50.573864 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182502c0-8250-47ab-b833-6e4826573eb4 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:50.579048 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:50.579720 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:50.579994 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:50.580263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] PATH_INFO: `/v3/projects/cd1f835336b64fcc9c850f69c4d8b711` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:50.581559 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd1f835336b64fcc9c850f69c4d8b711)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:50.583465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:50.634252 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd1f835336b64fcc9c850f69c4d8b711', 'request_id': 'req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.644885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd1f835336b64fcc9c850f69c4d8b711', 'request_id': 'req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.652388 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd1f835336b64fcc9c850f69c4d8b711', 'request_id': 'req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.657046 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:50.665108 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] The token cache is being invalidate because project cd1f835336b64fcc9c850f69c4d8b711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:50.665405 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.666316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.668050 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1375/2747] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:50 2022] DELETE /identity/v3/projects/cd1f835336b64fcc9c850f69c4d8b711 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:50.672300 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497d9af-afa2-4572-9492-922874986690 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:50.682265 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a497d9af-afa2-4572-9492-922874986690 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:50.748029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750906d4-fd7c-4016-bb2a-c89461a3c7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:50.748803 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750906d4-fd7c-4016-bb2a-c89461a3c7e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:50.749077 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750906d4-fd7c-4016-bb2a-c89461a3c7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:50.749347 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750906d4-fd7c-4016-bb2a-c89461a3c7e0 admin admin] PATH_INFO: `/v3/users/87f3373fbdf246b8bd7d6b319c8d3e3a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:50.750531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750906d4-fd7c-4016-bb2a-c89461a3c7e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f3373fbdf246b8bd7d6b319c8d3e3a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:50.752428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750906d4-fd7c-4016-bb2a-c89461a3c7e0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:50.804444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-750906d4-fd7c-4016-bb2a-c89461a3c7e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f3373fbdf246b8bd7d6b319c8d3e3a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.811467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-750906d4-fd7c-4016-bb2a-c89461a3c7e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f3373fbdf246b8bd7d6b319c8d3e3a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.824402 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-750906d4-fd7c-4016-bb2a-c89461a3c7e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f3373fbdf246b8bd7d6b319c8d3e3a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.835289 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-750906d4-fd7c-4016-bb2a-c89461a3c7e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f3373fbdf246b8bd7d6b319c8d3e3a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.835996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-750906d4-fd7c-4016-bb2a-c89461a3c7e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f3373fbdf246b8bd7d6b319c8d3e3a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:50.842191 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1373/2748] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:50 2022] DELETE /identity/v3/users/87f3373fbdf246b8bd7d6b319c8d3e3a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:51.145208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:51.146831 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:51.164938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c82eb7-ff79-4fd3-b5dd-c3297e561e26 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:51.227381 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50af0f-bcc8-4878-9ad7-54600faea24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:51.227903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50af0f-bcc8-4878-9ad7-54600faea24b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:51.228203 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50af0f-bcc8-4878-9ad7-54600faea24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:51.228462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50af0f-bcc8-4878-9ad7-54600faea24b admin admin] PATH_INFO: `/v3/projects/c997f98799164e8f86d3eb7a7c6e6a16` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:51.229790 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50af0f-bcc8-4878-9ad7-54600faea24b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c997f98799164e8f86d3eb7a7c6e6a16)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:51.231756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50af0f-bcc8-4878-9ad7-54600faea24b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:51.282487 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7f50af0f-bcc8-4878-9ad7-54600faea24b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c997f98799164e8f86d3eb7a7c6e6a16', 'request_id': 'req-7f50af0f-bcc8-4878-9ad7-54600faea24b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:51.291648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7f50af0f-bcc8-4878-9ad7-54600faea24b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c997f98799164e8f86d3eb7a7c6e6a16', 'request_id': 'req-7f50af0f-bcc8-4878-9ad7-54600faea24b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:51.301037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7f50af0f-bcc8-4878-9ad7-54600faea24b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c997f98799164e8f86d3eb7a7c6e6a16', 'request_id': 'req-7f50af0f-bcc8-4878-9ad7-54600faea24b'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:51.314150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7f50af0f-bcc8-4878-9ad7-54600faea24b admin admin] The token cache is being invalidate because project c997f98799164e8f86d3eb7a7c6e6a16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:51.314437 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7f50af0f-bcc8-4878-9ad7-54600faea24b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:51.315102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-7f50af0f-bcc8-4878-9ad7-54600faea24b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:51.316655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1376/2749] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:51 2022] DELETE /identity/v3/projects/c997f98799164e8f86d3eb7a7c6e6a16 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:51.322567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614fc3d-6e72-4d3f-97ec-1698b68b471b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:51.322853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614fc3d-6e72-4d3f-97ec-1698b68b471b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:51.323131 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614fc3d-6e72-4d3f-97ec-1698b68b471b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:51.346314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-8614fc3d-6e72-4d3f-97ec-1698b68b471b None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:51.347552 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8614fc3d-6e72-4d3f-97ec-1698b68b471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:51.406447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:51.421330 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1374/2750] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:28:51.434235 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b9675-5ca1-4d5a-8491-c489ca0d173a None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:51.434451 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b9675-5ca1-4d5a-8491-c489ca0d173a None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:51.434693 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b9675-5ca1-4d5a-8491-c489ca0d173a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:51.455756 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-596b9675-5ca1-4d5a-8491-c489ca0d173a None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:51.456990 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-596b9675-5ca1-4d5a-8491-c489ca0d173a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:51.465876 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1377/2751] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:51.470825 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614fc3d-6e72-4d3f-97ec-1698b68b471b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:51.484007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614fc3d-6e72-4d3f-97ec-1698b68b471b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:51.489685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b9c78-c080-4ef9-94ae-6fe973758f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:51.490293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b9c78-c080-4ef9-94ae-6fe973758f07 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:51.490563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b9c78-c080-4ef9-94ae-6fe973758f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:51.490826 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b9c78-c080-4ef9-94ae-6fe973758f07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:51.491481 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b9c78-c080-4ef9-94ae-6fe973758f07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:28:51.491782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b9c78-c080-4ef9-94ae-6fe973758f07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:51.493419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b9c78-c080-4ef9-94ae-6fe973758f07 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:51.506539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1375/2752] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:28:51.511394 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b9675-5ca1-4d5a-8491-c489ca0d173a None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:51.521103 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b9675-5ca1-4d5a-8491-c489ca0d173a None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:51.526902 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c954e-c18b-40da-91da-7668b0a8f296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:51.527506 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c954e-c18b-40da-91da-7668b0a8f296 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:51.527807 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c954e-c18b-40da-91da-7668b0a8f296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:51.528092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c954e-c18b-40da-91da-7668b0a8f296 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:51.528729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c954e-c18b-40da-91da-7668b0a8f296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:51.530659 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c954e-c18b-40da-91da-7668b0a8f296 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:51.564709 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1378/2753] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:28:51 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:51.569160 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b9c78-c080-4ef9-94ae-6fe973758f07 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:51.580038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b9c78-c080-4ef9-94ae-6fe973758f07 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:51.639553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bd59c-a795-4e94-9958-619b032aba88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:51.640151 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bd59c-a795-4e94-9958-619b032aba88 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:51.640424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bd59c-a795-4e94-9958-619b032aba88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:51.640734 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5bd59c-a795-4e94-9958-619b032aba88 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:51.641349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5bd59c-a795-4e94-9958-619b032aba88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:51.643227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5bd59c-a795-4e94-9958-619b032aba88 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:51.670509 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1376/2754] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:28:51 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:28:51.675627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c954e-c18b-40da-91da-7668b0a8f296 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:51.691337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c954e-c18b-40da-91da-7668b0a8f296 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:51.699830 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba011d-e19e-4401-b2a8-c55df4fb8340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:51.700436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba011d-e19e-4401-b2a8-c55df4fb8340 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:51.700740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba011d-e19e-4401-b2a8-c55df4fb8340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:51.701004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba011d-e19e-4401-b2a8-c55df4fb8340 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:51.701652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba011d-e19e-4401-b2a8-c55df4fb8340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:51.703259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba011d-e19e-4401-b2a8-c55df4fb8340 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:51.713882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1379/2755] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:51.718286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bd59c-a795-4e94-9958-619b032aba88 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:51.730932 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5bd59c-a795-4e94-9958-619b032aba88 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:51.737328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c8250-594f-4a61-8d32-69c43bbe7f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:51.737853 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c8250-594f-4a61-8d32-69c43bbe7f86 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:51.738125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c8250-594f-4a61-8d32-69c43bbe7f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:51.738394 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c8250-594f-4a61-8d32-69c43bbe7f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:51.739033 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c8250-594f-4a61-8d32-69c43bbe7f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:51.740624 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c8250-594f-4a61-8d32-69c43bbe7f86 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:51.752150 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1377/2756] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:51.757289 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba011d-e19e-4401-b2a8-c55df4fb8340 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:51.769781 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba011d-e19e-4401-b2a8-c55df4fb8340 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:51.775580 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebbcb6-55b7-40da-82ca-876fc446483e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:51.776224 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebbcb6-55b7-40da-82ca-876fc446483e admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:51.776496 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebbcb6-55b7-40da-82ca-876fc446483e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:51.776883 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebbcb6-55b7-40da-82ca-876fc446483e admin admin] PATH_INFO: `/v3/projects/565e4129e315483db086b7741866a85b/users/5c24862af430405ca6b25b836ac7b369/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:51.793894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ebbcb6-55b7-40da-82ca-876fc446483e admin admin] RBAC: Authorizing `identity:create_grant(project_id=565e4129e315483db086b7741866a85b, user_id=5c24862af430405ca6b25b836ac7b369, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:51.796535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ebbcb6-55b7-40da-82ca-876fc446483e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:51.815441 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1380/2757] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:51 2022] PUT /identity/v3/projects/565e4129e315483db086b7741866a85b/users/5c24862af430405ca6b25b836ac7b369/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:51.829555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f816627-3ce8-413b-90fe-52a69ae14bab None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:51.829844 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f816627-3ce8-413b-90fe-52a69ae14bab None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:51.830109 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f816627-3ce8-413b-90fe-52a69ae14bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:51.860947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-2f816627-3ce8-413b-90fe-52a69ae14bab None None] MFA Rules not processed for user `5c24862af430405ca6b25b836ac7b369`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:51.862195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-2f816627-3ce8-413b-90fe-52a69ae14bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:51.930809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1378/2758] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:51.936093 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebbcb6-55b7-40da-82ca-876fc446483e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:51.961422 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:52.001444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebbcb6-55b7-40da-82ca-876fc446483e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:52.006479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a3b38-e0bc-40af-b272-272a16634c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:52.007091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a3b38-e0bc-40af-b272-272a16634c47 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:52.007369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a3b38-e0bc-40af-b272-272a16634c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:52.007638 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a3b38-e0bc-40af-b272-272a16634c47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:52.008275 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a3b38-e0bc-40af-b272-272a16634c47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:52.010225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a3b38-e0bc-40af-b272-272a16634c47 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:52.038008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1381/2759] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:28:51 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:52.044491 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f816627-3ce8-413b-90fe-52a69ae14bab None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:52.055850 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f816627-3ce8-413b-90fe-52a69ae14bab None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:52.123659 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:52.135504 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293734f-9db2-4f66-9a1d-2305d6f42c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:52.136112 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293734f-9db2-4f66-9a1d-2305d6f42c00 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:52.136382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293734f-9db2-4f66-9a1d-2305d6f42c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:52.136673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293734f-9db2-4f66-9a1d-2305d6f42c00 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:52.137379 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293734f-9db2-4f66-9a1d-2305d6f42c00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:52.139662 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293734f-9db2-4f66-9a1d-2305d6f42c00 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:52.163317 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1379/2760] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:28:52 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:52.167551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a3b38-e0bc-40af-b272-272a16634c47 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:52.178040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a3b38-e0bc-40af-b272-272a16634c47 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:52.183211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9badb-bb32-4608-a8d1-0c011aeee534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:52.183866 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9badb-bb32-4608-a8d1-0c011aeee534 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:52.184143 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9badb-bb32-4608-a8d1-0c011aeee534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:52.184413 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9badb-bb32-4608-a8d1-0c011aeee534 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:52.185095 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9badb-bb32-4608-a8d1-0c011aeee534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:52.186693 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9badb-bb32-4608-a8d1-0c011aeee534 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:52.198008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1382/2761] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:52.201834 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293734f-9db2-4f66-9a1d-2305d6f42c00 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:52.214915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293734f-9db2-4f66-9a1d-2305d6f42c00 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:52.223141 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff93ce8-56a2-4f04-ad4f-30a5f09629d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:52.223947 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff93ce8-56a2-4f04-ad4f-30a5f09629d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:52.224344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff93ce8-56a2-4f04-ad4f-30a5f09629d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:52.224808 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff93ce8-56a2-4f04-ad4f-30a5f09629d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:52.225802 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff93ce8-56a2-4f04-ad4f-30a5f09629d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:52.228290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff93ce8-56a2-4f04-ad4f-30a5f09629d3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:52.240675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1380/2762] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:52.245156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9badb-bb32-4608-a8d1-0c011aeee534 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:52.256092 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9badb-bb32-4608-a8d1-0c011aeee534 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:52.261289 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de50ec-bb79-4e8d-a90c-f306e3a811b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:52.262024 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de50ec-bb79-4e8d-a90c-f306e3a811b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:52.262294 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de50ec-bb79-4e8d-a90c-f306e3a811b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:52.262555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de50ec-bb79-4e8d-a90c-f306e3a811b6 admin admin] PATH_INFO: `/v3/projects/0cea37607e0a4c5d817cffcd0c9df31e/users/11b11d408738407ab4aff46658876077/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:52.278011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de50ec-bb79-4e8d-a90c-f306e3a811b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cea37607e0a4c5d817cffcd0c9df31e, user_id=11b11d408738407ab4aff46658876077, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:52.280735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de50ec-bb79-4e8d-a90c-f306e3a811b6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:52.296490 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1383/2763] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:52 2022] PUT /identity/v3/projects/0cea37607e0a4c5d817cffcd0c9df31e/users/11b11d408738407ab4aff46658876077/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:28:52.307074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ffa6f-6538-4b72-81a2-17241233b051 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:52.307349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ffa6f-6538-4b72-81a2-17241233b051 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:52.307641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ffa6f-6538-4b72-81a2-17241233b051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:52.336664 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-e43ffa6f-6538-4b72-81a2-17241233b051 None None] MFA Rules not processed for user `11b11d408738407ab4aff46658876077`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:52.337925 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e43ffa6f-6538-4b72-81a2-17241233b051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:52.405876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1381/2764] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:28:52.410704 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de50ec-bb79-4e8d-a90c-f306e3a811b6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:52.479744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de50ec-bb79-4e8d-a90c-f306e3a811b6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:52.485537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9aec97-6d6b-4cbe-8fb8-9879c523dbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:52.486165 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9aec97-6d6b-4cbe-8fb8-9879c523dbbf admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:52.486436 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9aec97-6d6b-4cbe-8fb8-9879c523dbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:52.486701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9aec97-6d6b-4cbe-8fb8-9879c523dbbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:52.487334 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9aec97-6d6b-4cbe-8fb8-9879c523dbbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:52.489305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9aec97-6d6b-4cbe-8fb8-9879c523dbbf admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:52.520528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1384/2765] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:28:52 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:52.524450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ffa6f-6538-4b72-81a2-17241233b051 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:52.535847 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ffa6f-6538-4b72-81a2-17241233b051 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:52.594306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e948c-378d-4b99-b683-8b85779be75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:52.594914 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e948c-378d-4b99-b683-8b85779be75b admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:52.595189 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e948c-378d-4b99-b683-8b85779be75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:52.595459 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e948c-378d-4b99-b683-8b85779be75b admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:52.596110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e948c-378d-4b99-b683-8b85779be75b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:52.598044 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e948c-378d-4b99-b683-8b85779be75b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:52.624094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1382/2766] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:28:52 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:52.629859 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9aec97-6d6b-4cbe-8fb8-9879c523dbbf admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:52.642261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9aec97-6d6b-4cbe-8fb8-9879c523dbbf admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:52.648630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fafc5-d1d9-460c-8941-6e5fbc94b8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:52.649211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fafc5-d1d9-460c-8941-6e5fbc94b8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:52.649486 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fafc5-d1d9-460c-8941-6e5fbc94b8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:52.649749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fafc5-d1d9-460c-8941-6e5fbc94b8d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:52.650395 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0fafc5-d1d9-460c-8941-6e5fbc94b8d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:52.651992 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0fafc5-d1d9-460c-8941-6e5fbc94b8d5 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:52.663805 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1385/2767] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:28:52.668285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e948c-378d-4b99-b683-8b85779be75b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:52.679524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e948c-378d-4b99-b683-8b85779be75b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:52.685623 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8ae4a-8d89-47cd-a231-639a4d216bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:52.686255 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8ae4a-8d89-47cd-a231-639a4d216bda admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:52.686508 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8ae4a-8d89-47cd-a231-639a4d216bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:52.686809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b8ae4a-8d89-47cd-a231-639a4d216bda admin admin] PATH_INFO: `/v3/projects/a081a74fe4b848bfa36bb94a110fa6e9/users/245de8609a47446682a81d4ef1843b0b/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:52.702247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8ae4a-8d89-47cd-a231-639a4d216bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=a081a74fe4b848bfa36bb94a110fa6e9, user_id=245de8609a47446682a81d4ef1843b0b, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:52.705022 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b8ae4a-8d89-47cd-a231-639a4d216bda admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:52.720551 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1383/2768] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:52 2022] PUT /identity/v3/projects/a081a74fe4b848bfa36bb94a110fa6e9/users/245de8609a47446682a81d4ef1843b0b/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:52.724624 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fafc5-d1d9-460c-8941-6e5fbc94b8d5 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:52.736534 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fafc5-d1d9-460c-8941-6e5fbc94b8d5 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:52.790427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:52.806701 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34b789-0c6d-4ef3-873d-edaa34973a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:52.807303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34b789-0c6d-4ef3-873d-edaa34973a93 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:52.807592 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34b789-0c6d-4ef3-873d-edaa34973a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:52.807844 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34b789-0c6d-4ef3-873d-edaa34973a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:52.808491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34b789-0c6d-4ef3-873d-edaa34973a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:52.810806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34b789-0c6d-4ef3-873d-edaa34973a93 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:52.820617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1386/2769] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:52.824890 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8ae4a-8d89-47cd-a231-639a4d216bda admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:52.840964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b8ae4a-8d89-47cd-a231-639a4d216bda admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:52.847025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f0631-20a8-4068-81e9-edc316ba9a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:52.847711 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f0631-20a8-4068-81e9-edc316ba9a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:52.847983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f0631-20a8-4068-81e9-edc316ba9a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:52.848249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f0631-20a8-4068-81e9-edc316ba9a68 admin admin] PATH_INFO: `/v3/projects/a081a74fe4b848bfa36bb94a110fa6e9/users/245de8609a47446682a81d4ef1843b0b/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:52.850469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f0631-20a8-4068-81e9-edc316ba9a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a081a74fe4b848bfa36bb94a110fa6e9, user_id=245de8609a47446682a81d4ef1843b0b, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:52.853170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f0631-20a8-4068-81e9-edc316ba9a68 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:52.864548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1384/2770] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:52 2022] PUT /identity/v3/projects/a081a74fe4b848bfa36bb94a110fa6e9/users/245de8609a47446682a81d4ef1843b0b/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:52.876456 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5b274-5fa7-4311-adfa-4a6cc05faccc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:52.876779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5b274-5fa7-4311-adfa-4a6cc05faccc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:52.877050 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5b274-5fa7-4311-adfa-4a6cc05faccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:52.905890 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-67d5b274-5fa7-4311-adfa-4a6cc05faccc None None] MFA Rules not processed for user `245de8609a47446682a81d4ef1843b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:52.907082 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-67d5b274-5fa7-4311-adfa-4a6cc05faccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:52.980543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1387/2771] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:52.992245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f0631-20a8-4068-81e9-edc316ba9a68 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:52.994430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-918f0631-20a8-4068-81e9-edc316ba9a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:53.010668 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:53.058307 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f0631-20a8-4068-81e9-edc316ba9a68 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:53.062094 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf409f-8e55-4c59-8930-410726e0fb16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:53.062689 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf409f-8e55-4c59-8930-410726e0fb16 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:53.062958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf409f-8e55-4c59-8930-410726e0fb16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:53.063218 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf409f-8e55-4c59-8930-410726e0fb16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:53.072758 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf409f-8e55-4c59-8930-410726e0fb16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:53.074192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf409f-8e55-4c59-8930-410726e0fb16 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:53.083963 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1385/2772] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:53.140929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5b274-5fa7-4311-adfa-4a6cc05faccc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:53.143555 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-67d5b274-5fa7-4311-adfa-4a6cc05faccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:53.164377 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5b274-5fa7-4311-adfa-4a6cc05faccc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:53.173792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ae0db-5056-42c6-9f04-f3a62af562e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:53.174419 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ae0db-5056-42c6-9f04-f3a62af562e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:53.174690 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ae0db-5056-42c6-9f04-f3a62af562e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:53.174954 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ae0db-5056-42c6-9f04-f3a62af562e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:53.237669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ae0db-5056-42c6-9f04-f3a62af562e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:53.239102 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ae0db-5056-42c6-9f04-f3a62af562e7 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:53.248503 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1388/2773] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:56.244467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf409f-8e55-4c59-8930-410726e0fb16 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:56.319982 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf409f-8e55-4c59-8930-410726e0fb16 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:56.324531 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:56.325181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:56.325447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:56.325708 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] PATH_INFO: `/v3/users/5c24862af430405ca6b25b836ac7b369` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:56.326801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c24862af430405ca6b25b836ac7b369)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:56.328873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:56.381320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c24862af430405ca6b25b836ac7b369'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:56.387989 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c24862af430405ca6b25b836ac7b369'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:56.401080 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c24862af430405ca6b25b836ac7b369'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:56.410416 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c24862af430405ca6b25b836ac7b369'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:56.411080 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c24862af430405ca6b25b836ac7b369'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:56.417515 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1386/2774] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:56 2022] DELETE /identity/v3/users/5c24862af430405ca6b25b836ac7b369 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:56.423194 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df85f2-2682-491b-9574-bfc7170e31cd None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:56.423739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df85f2-2682-491b-9574-bfc7170e31cd None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:56.423739 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df85f2-2682-491b-9574-bfc7170e31cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:56.443002 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-05df85f2-2682-491b-9574-bfc7170e31cd None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:56.444196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-05df85f2-2682-491b-9574-bfc7170e31cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:56.506070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1389/2775] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:56.516582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:56.518123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:56.555501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:56.582177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17f7f6-14c4-4948-8152-d9d9720d46b3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:56.586090 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a97e1a-945f-4b2e-b92e-4a1c1ce52bb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:56.586698 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a97e1a-945f-4b2e-b92e-4a1c1ce52bb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:56.586972 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a97e1a-945f-4b2e-b92e-4a1c1ce52bb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:56.587238 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a97e1a-945f-4b2e-b92e-4a1c1ce52bb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:56.596829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a97e1a-945f-4b2e-b92e-4a1c1ce52bb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:56.598232 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a97e1a-945f-4b2e-b92e-4a1c1ce52bb3 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:56.607432 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1387/2776] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:28:56.880724 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df85f2-2682-491b-9574-bfc7170e31cd None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:56.882293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-05df85f2-2682-491b-9574-bfc7170e31cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:56.895208 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df85f2-2682-491b-9574-bfc7170e31cd None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:56.901005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:56.901795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:56.902129 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:56.902466 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] PATH_INFO: `/v3/projects/565e4129e315483db086b7741866a85b` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:56.903775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=565e4129e315483db086b7741866a85b)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:56.905924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:56.908639 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=100 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:56.957125 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '565e4129e315483db086b7741866a85b', 'request_id': 'req-558cb53d-757f-41a3-aa28-a7fa4da5aa88'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:56.966489 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '565e4129e315483db086b7741866a85b', 'request_id': 'req-558cb53d-757f-41a3-aa28-a7fa4da5aa88'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:56.972920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '565e4129e315483db086b7741866a85b', 'request_id': 'req-558cb53d-757f-41a3-aa28-a7fa4da5aa88'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:56.983199 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] The token cache is being invalidate because project 565e4129e315483db086b7741866a85b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:56.983483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:56.984138 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:56.985692 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1390/2777] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:56 2022] DELETE /identity/v3/projects/565e4129e315483db086b7741866a85b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:28:56.989829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a97e1a-945f-4b2e-b92e-4a1c1ce52bb3 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:56.999978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a97e1a-945f-4b2e-b92e-4a1c1ce52bb3 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:57.066387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:57.067018 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:57.067293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:57.067561 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] PATH_INFO: `/v3/users/11b11d408738407ab4aff46658876077` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:57.068795 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b11d408738407ab4aff46658876077)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:57.070751 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:57.123467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b11d408738407ab4aff46658876077'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.130473 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b11d408738407ab4aff46658876077'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.142419 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b11d408738407ab4aff46658876077'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.152505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b11d408738407ab4aff46658876077'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.153158 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b11d408738407ab4aff46658876077'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.159900 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1388/2778] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:56 2022] DELETE /identity/v3/users/11b11d408738407ab4aff46658876077 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:57.471783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:57.473492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:57.495960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cb53d-757f-41a3-aa28-a7fa4da5aa88 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:57.556338 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:57.557158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:57.557422 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:57.557699 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] PATH_INFO: `/v3/projects/0cea37607e0a4c5d817cffcd0c9df31e` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:57.558898 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cea37607e0a4c5d817cffcd0c9df31e)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:57.560855 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:57.609951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cea37607e0a4c5d817cffcd0c9df31e', 'request_id': 'req-f44bd467-db88-45c0-8152-23c6cd12a243'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.618840 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cea37607e0a4c5d817cffcd0c9df31e', 'request_id': 'req-f44bd467-db88-45c0-8152-23c6cd12a243'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.626020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cea37607e0a4c5d817cffcd0c9df31e', 'request_id': 'req-f44bd467-db88-45c0-8152-23c6cd12a243'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.637150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] The token cache is being invalidate because project 0cea37607e0a4c5d817cffcd0c9df31e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:57.637428 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.638073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.639576 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1391/2779] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:57 2022] DELETE /identity/v3/projects/0cea37607e0a4c5d817cffcd0c9df31e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:57.643829 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:57.645366 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:57.658518 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16228eb-e5c8-41a5-8ff0-c75f0b912ad6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:57.710315 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:57.719996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88c00c-3b07-45c4-aca0-a55680d7f6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:57.720571 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c00c-3b07-45c4-aca0-a55680d7f6d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:57.720884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c00c-3b07-45c4-aca0-a55680d7f6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:57.721161 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88c00c-3b07-45c4-aca0-a55680d7f6d0 admin admin] PATH_INFO: `/v3/users/245de8609a47446682a81d4ef1843b0b` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:57.722415 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88c00c-3b07-45c4-aca0-a55680d7f6d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=245de8609a47446682a81d4ef1843b0b)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:57.724653 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88c00c-3b07-45c4-aca0-a55680d7f6d0 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:57.778245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4b88c00c-3b07-45c4-aca0-a55680d7f6d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '245de8609a47446682a81d4ef1843b0b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.785399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4b88c00c-3b07-45c4-aca0-a55680d7f6d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '245de8609a47446682a81d4ef1843b0b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.798352 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4b88c00c-3b07-45c4-aca0-a55680d7f6d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '245de8609a47446682a81d4ef1843b0b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.806500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4b88c00c-3b07-45c4-aca0-a55680d7f6d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '245de8609a47446682a81d4ef1843b0b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.807129 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-4b88c00c-3b07-45c4-aca0-a55680d7f6d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '245de8609a47446682a81d4ef1843b0b'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:57.813398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1389/2780] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:28:57 2022] DELETE /identity/v3/users/245de8609a47446682a81d4ef1843b0b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:58.095361 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:58.097328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:58.113250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44bd467-db88-45c0-8152-23c6cd12a243 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:58.131919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:58.180039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352a942-6ddd-4ca0-8fb7-05b43030093f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:58.180696 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352a942-6ddd-4ca0-8fb7-05b43030093f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:58.180975 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352a942-6ddd-4ca0-8fb7-05b43030093f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:58.181245 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352a942-6ddd-4ca0-8fb7-05b43030093f admin admin] PATH_INFO: `/v3/projects/a081a74fe4b848bfa36bb94a110fa6e9` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:58.182433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352a942-6ddd-4ca0-8fb7-05b43030093f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a081a74fe4b848bfa36bb94a110fa6e9)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:58.184391 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352a942-6ddd-4ca0-8fb7-05b43030093f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:58.234997 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0352a942-6ddd-4ca0-8fb7-05b43030093f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a081a74fe4b848bfa36bb94a110fa6e9', 'request_id': 'req-0352a942-6ddd-4ca0-8fb7-05b43030093f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:58.243851 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0352a942-6ddd-4ca0-8fb7-05b43030093f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a081a74fe4b848bfa36bb94a110fa6e9', 'request_id': 'req-0352a942-6ddd-4ca0-8fb7-05b43030093f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:58.251037 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0352a942-6ddd-4ca0-8fb7-05b43030093f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a081a74fe4b848bfa36bb94a110fa6e9', 'request_id': 'req-0352a942-6ddd-4ca0-8fb7-05b43030093f'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:58.261600 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0352a942-6ddd-4ca0-8fb7-05b43030093f admin admin] The token cache is being invalidate because project a081a74fe4b848bfa36bb94a110fa6e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:28:58.261823 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0352a942-6ddd-4ca0-8fb7-05b43030093f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:58.262452 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0352a942-6ddd-4ca0-8fb7-05b43030093f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:28:58.263929 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1392/2781] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:28:58 2022] DELETE /identity/v3/projects/a081a74fe4b848bfa36bb94a110fa6e9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:58.271502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9568b-30a2-4c28-8479-290d824a990d None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:58.271781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9568b-30a2-4c28-8479-290d824a990d None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:58.272079 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9568b-30a2-4c28-8479-290d824a990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:58.292323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-25a9568b-30a2-4c28-8479-290d824a990d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:58.293568 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-25a9568b-30a2-4c28-8479-290d824a990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:58.353512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1390/2782] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:58.363131 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d53154-d663-4de2-bae4-491de5e2e1cc None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:58.363522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d53154-d663-4de2-bae4-491de5e2e1cc None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:58.363880 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d53154-d663-4de2-bae4-491de5e2e1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:58.388946 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-01d53154-d663-4de2-bae4-491de5e2e1cc None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:58.390156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-01d53154-d663-4de2-bae4-491de5e2e1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:58.398686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1393/2783] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:28:58.404190 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9568b-30a2-4c28-8479-290d824a990d None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:58.414117 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9568b-30a2-4c28-8479-290d824a990d None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:58.419892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdb980-e978-45dd-9a33-de5210b92434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:58.420499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdb980-e978-45dd-9a33-de5210b92434 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:58.420814 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdb980-e978-45dd-9a33-de5210b92434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:58.421082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdb980-e978-45dd-9a33-de5210b92434 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:58.421731 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdb980-e978-45dd-9a33-de5210b92434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:28:58.422030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdb980-e978-45dd-9a33-de5210b92434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:58.423685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdb980-e978-45dd-9a33-de5210b92434 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:58.436258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1391/2784] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:28:58.440870 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d53154-d663-4de2-bae4-491de5e2e1cc None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:58.450439 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d53154-d663-4de2-bae4-491de5e2e1cc None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:58.456012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcded03-9de0-4990-bcba-c9df1330f096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:58.456651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcded03-9de0-4990-bcba-c9df1330f096 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:58.456924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcded03-9de0-4990-bcba-c9df1330f096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:58.457191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcded03-9de0-4990-bcba-c9df1330f096 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:58.457817 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcded03-9de0-4990-bcba-c9df1330f096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:58.459702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcded03-9de0-4990-bcba-c9df1330f096 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:58.489029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1394/2785] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:28:58 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:58.493275 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdb980-e978-45dd-9a33-de5210b92434 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:58.504389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdb980-e978-45dd-9a33-de5210b92434 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:58.563564 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3042e6-89cb-4eb5-9052-d44fa100c70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:58.564154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3042e6-89cb-4eb5-9052-d44fa100c70e admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:58.564426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3042e6-89cb-4eb5-9052-d44fa100c70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:58.564721 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3042e6-89cb-4eb5-9052-d44fa100c70e admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:58.565348 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3042e6-89cb-4eb5-9052-d44fa100c70e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:58.567250 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3042e6-89cb-4eb5-9052-d44fa100c70e admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:58.576119 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:58.592212 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1392/2786] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:28:58 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:28:58.595767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcded03-9de0-4990-bcba-c9df1330f096 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:58.606107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcded03-9de0-4990-bcba-c9df1330f096 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:58.611039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e3fac-30aa-4478-a709-32175c5619be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:58.611637 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e3fac-30aa-4478-a709-32175c5619be admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:58.611915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e3fac-30aa-4478-a709-32175c5619be admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:58.612178 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e3fac-30aa-4478-a709-32175c5619be admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:58.612858 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e3fac-30aa-4478-a709-32175c5619be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:58.614459 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e3fac-30aa-4478-a709-32175c5619be admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:58.624700 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1395/2787] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:58.628940 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3042e6-89cb-4eb5-9052-d44fa100c70e admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:58.639163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3042e6-89cb-4eb5-9052-d44fa100c70e admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:58.644608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a30a67-0a1a-48b6-ab96-9972071b8c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:58.645227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a30a67-0a1a-48b6-ab96-9972071b8c95 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:58.645540 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a30a67-0a1a-48b6-ab96-9972071b8c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:58.645784 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a30a67-0a1a-48b6-ab96-9972071b8c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:58.646432 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a30a67-0a1a-48b6-ab96-9972071b8c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:58.648011 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a30a67-0a1a-48b6-ab96-9972071b8c95 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:58.659968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1393/2788] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:28:58.664469 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e3fac-30aa-4478-a709-32175c5619be admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:58.675660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e3fac-30aa-4478-a709-32175c5619be admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:58.681190 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715be525-a25e-4ba9-824d-4d53bacdff91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:58.681836 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715be525-a25e-4ba9-824d-4d53bacdff91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:58.682112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715be525-a25e-4ba9-824d-4d53bacdff91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:58.682380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715be525-a25e-4ba9-824d-4d53bacdff91 admin admin] PATH_INFO: `/v3/projects/4e94c5e027104aef957c6387bd80c25e/users/4378050be19d4eafa9298a88440598e1/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:58.697336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715be525-a25e-4ba9-824d-4d53bacdff91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e94c5e027104aef957c6387bd80c25e, user_id=4378050be19d4eafa9298a88440598e1, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:58.700025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715be525-a25e-4ba9-824d-4d53bacdff91 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:58.716679 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1396/2789] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:28:58 2022] PUT /identity/v3/projects/4e94c5e027104aef957c6387bd80c25e/users/4378050be19d4eafa9298a88440598e1/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:28:58.729225 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502286d-7003-4985-ac4d-afc09de3625e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:58.729506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502286d-7003-4985-ac4d-afc09de3625e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:58.729772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502286d-7003-4985-ac4d-afc09de3625e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:58.760507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5502286d-7003-4985-ac4d-afc09de3625e None None] MFA Rules not processed for user `4378050be19d4eafa9298a88440598e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:28:58.761746 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5502286d-7003-4985-ac4d-afc09de3625e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:58.830064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1394/2790] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:28:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:28:58.841535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715be525-a25e-4ba9-824d-4d53bacdff91 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:28:58.843351 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-715be525-a25e-4ba9-824d-4d53bacdff91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:28:58.907250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:28:58.913569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715be525-a25e-4ba9-824d-4d53bacdff91 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:28:58.918310 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2490838-01fb-4da9-acd7-1edd1034b34b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:28:58.918918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2490838-01fb-4da9-acd7-1edd1034b34b service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:28:58.919202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2490838-01fb-4da9-acd7-1edd1034b34b service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:28:58.919471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2490838-01fb-4da9-acd7-1edd1034b34b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:28:58.928962 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2490838-01fb-4da9-acd7-1edd1034b34b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:28:58.930393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2490838-01fb-4da9-acd7-1edd1034b34b service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:28:58.939740 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1397/2791] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:28:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:29:00.193764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502286d-7003-4985-ac4d-afc09de3625e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:00.195448 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5502286d-7003-4985-ac4d-afc09de3625e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:00.213260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502286d-7003-4985-ac4d-afc09de3625e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:00.220497 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7852c3-90e1-40ae-a00e-6e5b19ebcaca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:00.221524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7852c3-90e1-40ae-a00e-6e5b19ebcaca service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:00.221956 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7852c3-90e1-40ae-a00e-6e5b19ebcaca service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:00.222375 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7852c3-90e1-40ae-a00e-6e5b19ebcaca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:00.224810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cd7852c3-90e1-40ae-a00e-6e5b19ebcaca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:00.313634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7852c3-90e1-40ae-a00e-6e5b19ebcaca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:00.315062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7852c3-90e1-40ae-a00e-6e5b19ebcaca service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:00.334038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1395/2792] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:29:00 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:29:07.298167 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2490838-01fb-4da9-acd7-1edd1034b34b service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:07.300027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f2490838-01fb-4da9-acd7-1edd1034b34b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:07.373025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2490838-01fb-4da9-acd7-1edd1034b34b service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:07.377244 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcf3e8-3cbe-4188-94cf-6aaa8b1a1850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:07.377882 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcf3e8-3cbe-4188-94cf-6aaa8b1a1850 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:07.378135 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcf3e8-3cbe-4188-94cf-6aaa8b1a1850 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:07.378542 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcf3e8-3cbe-4188-94cf-6aaa8b1a1850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:07.380438 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a5dcf3e8-3cbe-4188-94cf-6aaa8b1a1850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:07.447750 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcf3e8-3cbe-4188-94cf-6aaa8b1a1850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:07.449133 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcf3e8-3cbe-4188-94cf-6aaa8b1a1850 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:07.458922 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1398/2793] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:29:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:29:09.116996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7852c3-90e1-40ae-a00e-6e5b19ebcaca service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:09.128894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7852c3-90e1-40ae-a00e-6e5b19ebcaca service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:09.134440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:09.135181 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:09.135556 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:09.135920 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] PATH_INFO: `/v3/users/4378050be19d4eafa9298a88440598e1` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:09.137196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4378050be19d4eafa9298a88440598e1)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:09.139197 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:09.184398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:29:09.196671 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4378050be19d4eafa9298a88440598e1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:09.204412 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4378050be19d4eafa9298a88440598e1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:09.220302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4378050be19d4eafa9298a88440598e1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:09.229955 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4378050be19d4eafa9298a88440598e1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:09.230609 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4378050be19d4eafa9298a88440598e1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:09.237497 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1396/2794] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:29:09 2022] DELETE /identity/v3/users/4378050be19d4eafa9298a88440598e1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:29:09.243630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67977720-a89b-4e34-8e95-e76a87c7cd83 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:09.244806 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67977720-a89b-4e34-8e95-e76a87c7cd83 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:09.245293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67977720-a89b-4e34-8e95-e76a87c7cd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:09.271954 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-67977720-a89b-4e34-8e95-e76a87c7cd83 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:09.273649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-67977720-a89b-4e34-8e95-e76a87c7cd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:09.347778 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1399/2795] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:29:09.357964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:09.359634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:09.427074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06a737-ef42-4d2b-91a8-f3a17f344fd6 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:09.432646 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338da239-0ad0-4fbe-a5a7-1998bc81c99a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:09.433579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338da239-0ad0-4fbe-a5a7-1998bc81c99a service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:09.433718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338da239-0ad0-4fbe-a5a7-1998bc81c99a service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:09.433868 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338da239-0ad0-4fbe-a5a7-1998bc81c99a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:09.445709 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338da239-0ad0-4fbe-a5a7-1998bc81c99a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:09.447154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338da239-0ad0-4fbe-a5a7-1998bc81c99a service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:09.457889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1397/2796] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:29:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:29:09.778917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67977720-a89b-4e34-8e95-e76a87c7cd83 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:09.780353 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-67977720-a89b-4e34-8e95-e76a87c7cd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:09.799530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67977720-a89b-4e34-8e95-e76a87c7cd83 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:09.805198 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a78b6-953a-49ce-afda-e2f57ddfa971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:09.805855 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a78b6-953a-49ce-afda-e2f57ddfa971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:09.806130 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a78b6-953a-49ce-afda-e2f57ddfa971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:09.806404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a78b6-953a-49ce-afda-e2f57ddfa971 admin admin] PATH_INFO: `/v3/projects/4e94c5e027104aef957c6387bd80c25e` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:09.807529 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a78b6-953a-49ce-afda-e2f57ddfa971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e94c5e027104aef957c6387bd80c25e)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:09.809585 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a78b6-953a-49ce-afda-e2f57ddfa971 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:09.849687 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:29:09.862293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-434a78b6-953a-49ce-afda-e2f57ddfa971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e94c5e027104aef957c6387bd80c25e', 'request_id': 'req-434a78b6-953a-49ce-afda-e2f57ddfa971'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:09.872187 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-434a78b6-953a-49ce-afda-e2f57ddfa971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e94c5e027104aef957c6387bd80c25e', 'request_id': 'req-434a78b6-953a-49ce-afda-e2f57ddfa971'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:09.879120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-434a78b6-953a-49ce-afda-e2f57ddfa971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e94c5e027104aef957c6387bd80c25e', 'request_id': 'req-434a78b6-953a-49ce-afda-e2f57ddfa971'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:09.890158 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-434a78b6-953a-49ce-afda-e2f57ddfa971 admin admin] The token cache is being invalidate because project 4e94c5e027104aef957c6387bd80c25e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:29:09.890444 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-434a78b6-953a-49ce-afda-e2f57ddfa971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:09.891085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-434a78b6-953a-49ce-afda-e2f57ddfa971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:09.892602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1400/2797] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:29:09 2022] DELETE /identity/v3/projects/4e94c5e027104aef957c6387bd80c25e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:29:09.898592 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4642efc-b6c9-49e0-882c-8d1143533c89 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:09.898922 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4642efc-b6c9-49e0-882c-8d1143533c89 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:09.899167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4642efc-b6c9-49e0-882c-8d1143533c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:09.920162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-c4642efc-b6c9-49e0-882c-8d1143533c89 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:09.921479 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c4642efc-b6c9-49e0-882c-8d1143533c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:09.983732 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1398/2798] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:29:09.991893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145000be-a4e7-4fc9-a5a8-632e4dbfc098 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:09.992251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145000be-a4e7-4fc9-a5a8-632e4dbfc098 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:09.992615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145000be-a4e7-4fc9-a5a8-632e4dbfc098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:10.014242 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-145000be-a4e7-4fc9-a5a8-632e4dbfc098 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:10.015543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-145000be-a4e7-4fc9-a5a8-632e4dbfc098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:10.024899 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1401/2799] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:29:10.029467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4642efc-b6c9-49e0-882c-8d1143533c89 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:10.039910 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4642efc-b6c9-49e0-882c-8d1143533c89 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:10.045092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ab10e-cea9-4d44-ae98-2b981b289a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:10.045693 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ab10e-cea9-4d44-ae98-2b981b289a1c admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:10.045986 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ab10e-cea9-4d44-ae98-2b981b289a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:10.046249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ab10e-cea9-4d44-ae98-2b981b289a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:10.046893 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ab10e-cea9-4d44-ae98-2b981b289a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:29:10.047185 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ab10e-cea9-4d44-ae98-2b981b289a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:10.049034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ab10e-cea9-4d44-ae98-2b981b289a1c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:10.060964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1399/2800] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:29:10.065230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145000be-a4e7-4fc9-a5a8-632e4dbfc098 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:10.075508 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145000be-a4e7-4fc9-a5a8-632e4dbfc098 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:10.080991 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc00c6-1f39-4337-9766-b65feb139f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:10.081595 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc00c6-1f39-4337-9766-b65feb139f5e admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:10.081866 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc00c6-1f39-4337-9766-b65feb139f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:10.082126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc00c6-1f39-4337-9766-b65feb139f5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:10.082774 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc00c6-1f39-4337-9766-b65feb139f5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:10.085590 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc00c6-1f39-4337-9766-b65feb139f5e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:10.123561 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1402/2801] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:29:10 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:29:10.127921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ab10e-cea9-4d44-ae98-2b981b289a1c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:10.139162 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ab10e-cea9-4d44-ae98-2b981b289a1c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:10.189879 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:29:10.220797 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194e4f5-5ea4-44a8-a50a-000b2e7ee99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:10.222125 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194e4f5-5ea4-44a8-a50a-000b2e7ee99f admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:10.222405 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194e4f5-5ea4-44a8-a50a-000b2e7ee99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:10.222672 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194e4f5-5ea4-44a8-a50a-000b2e7ee99f admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:10.223336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2194e4f5-5ea4-44a8-a50a-000b2e7ee99f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:10.225804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2194e4f5-5ea4-44a8-a50a-000b2e7ee99f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:10.251138 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1400/2802] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:29:10 2022] POST /identity/v3/users => generated 291 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:29:10.255350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc00c6-1f39-4337-9766-b65feb139f5e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:10.266729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc00c6-1f39-4337-9766-b65feb139f5e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:10.272772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6c517-e491-484c-8ba6-057f2b6a10ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:10.273805 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6c517-e491-484c-8ba6-057f2b6a10ea admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:10.274319 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6c517-e491-484c-8ba6-057f2b6a10ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:10.274669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6c517-e491-484c-8ba6-057f2b6a10ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:10.275373 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6c517-e491-484c-8ba6-057f2b6a10ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:10.277144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6c517-e491-484c-8ba6-057f2b6a10ea admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:10.288356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1403/2803] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:29:10.292896 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194e4f5-5ea4-44a8-a50a-000b2e7ee99f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:10.304439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194e4f5-5ea4-44a8-a50a-000b2e7ee99f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:10.311238 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003919f-6bdb-4fa7-9f35-3326168ce7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:10.312056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003919f-6bdb-4fa7-9f35-3326168ce7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:10.312296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003919f-6bdb-4fa7-9f35-3326168ce7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:10.312577 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003919f-6bdb-4fa7-9f35-3326168ce7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:10.313257 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003919f-6bdb-4fa7-9f35-3326168ce7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:10.314858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003919f-6bdb-4fa7-9f35-3326168ce7d7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:10.328954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1401/2804] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:29:10.333351 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6c517-e491-484c-8ba6-057f2b6a10ea admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:10.344281 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6c517-e491-484c-8ba6-057f2b6a10ea admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:10.349375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138eabbf-756e-4929-b822-7a99fe9f94e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:10.350221 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138eabbf-756e-4929-b822-7a99fe9f94e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:10.350492 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138eabbf-756e-4929-b822-7a99fe9f94e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:10.350758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138eabbf-756e-4929-b822-7a99fe9f94e8 admin admin] PATH_INFO: `/v3/projects/ace0c8e4f20649e28610cde631bc58b8/users/a63e79700c854a3f9581a339ee38376e/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:10.366328 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138eabbf-756e-4929-b822-7a99fe9f94e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ace0c8e4f20649e28610cde631bc58b8, user_id=a63e79700c854a3f9581a339ee38376e, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:10.369004 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138eabbf-756e-4929-b822-7a99fe9f94e8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:10.385017 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1404/2805] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:29:10 2022] PUT /identity/v3/projects/ace0c8e4f20649e28610cde631bc58b8/users/a63e79700c854a3f9581a339ee38376e/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:29:10.390221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bca7e-6601-4c8d-88af-8d6f423cea52 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:10.390523 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bca7e-6601-4c8d-88af-8d6f423cea52 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:10.390811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6bca7e-6601-4c8d-88af-8d6f423cea52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:10.410273 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-bc6bca7e-6601-4c8d-88af-8d6f423cea52 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:10.411469 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-bc6bca7e-6601-4c8d-88af-8d6f423cea52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:10.471753 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1402/2806] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:29:10.481891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138eabbf-756e-4929-b822-7a99fe9f94e8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:10.483888 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-138eabbf-756e-4929-b822-7a99fe9f94e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:10.564570 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138eabbf-756e-4929-b822-7a99fe9f94e8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:10.581269 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071bd95-002b-4804-8988-5caefd976b33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:10.582332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071bd95-002b-4804-8988-5caefd976b33 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:10.582332 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071bd95-002b-4804-8988-5caefd976b33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:10.582533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071bd95-002b-4804-8988-5caefd976b33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:10.604120 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071bd95-002b-4804-8988-5caefd976b33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:10.605628 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071bd95-002b-4804-8988-5caefd976b33 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:10.619081 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1405/2807] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:29:10.787554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bca7e-6601-4c8d-88af-8d6f423cea52 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:10.789195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-bc6bca7e-6601-4c8d-88af-8d6f423cea52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:10.803894 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6bca7e-6601-4c8d-88af-8d6f423cea52 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:10.809389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5db33-7510-4075-b71a-9002fc0b6d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:10.810097 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5db33-7510-4075-b71a-9002fc0b6d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:10.810374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5db33-7510-4075-b71a-9002fc0b6d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:10.810644 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5db33-7510-4075-b71a-9002fc0b6d60 admin admin] PATH_INFO: `/v3/users/e38b853b47b64567b00092eb32201b4d` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:10.811811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5db33-7510-4075-b71a-9002fc0b6d60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e38b853b47b64567b00092eb32201b4d)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:10.813795 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5db33-7510-4075-b71a-9002fc0b6d60 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:10.874611 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49b5db33-7510-4075-b71a-9002fc0b6d60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e38b853b47b64567b00092eb32201b4d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:10.882696 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49b5db33-7510-4075-b71a-9002fc0b6d60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e38b853b47b64567b00092eb32201b4d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:10.895138 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49b5db33-7510-4075-b71a-9002fc0b6d60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e38b853b47b64567b00092eb32201b4d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:10.904970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49b5db33-7510-4075-b71a-9002fc0b6d60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e38b853b47b64567b00092eb32201b4d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:10.905943 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-49b5db33-7510-4075-b71a-9002fc0b6d60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e38b853b47b64567b00092eb32201b4d'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:10.912582 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1403/2808] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:29:10 2022] DELETE /identity/v3/users/e38b853b47b64567b00092eb32201b4d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:29:11.126152 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071bd95-002b-4804-8988-5caefd976b33 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:11.148250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071bd95-002b-4804-8988-5caefd976b33 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:11.216775 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c130255-ab77-4ff5-82d4-24a440866e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:11.217584 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c130255-ab77-4ff5-82d4-24a440866e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:11.217864 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c130255-ab77-4ff5-82d4-24a440866e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:11.218173 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c130255-ab77-4ff5-82d4-24a440866e82 admin admin] PATH_INFO: `/v3/projects/bb9f84f2d16a4389a4be567cd2be3a8a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:11.219326 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c130255-ab77-4ff5-82d4-24a440866e82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb9f84f2d16a4389a4be567cd2be3a8a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:11.222069 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c130255-ab77-4ff5-82d4-24a440866e82 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:11.224811 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:29:11.282773 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5c130255-ab77-4ff5-82d4-24a440866e82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb9f84f2d16a4389a4be567cd2be3a8a', 'request_id': 'req-5c130255-ab77-4ff5-82d4-24a440866e82'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:11.292534 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5c130255-ab77-4ff5-82d4-24a440866e82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb9f84f2d16a4389a4be567cd2be3a8a', 'request_id': 'req-5c130255-ab77-4ff5-82d4-24a440866e82'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:11.299754 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5c130255-ab77-4ff5-82d4-24a440866e82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb9f84f2d16a4389a4be567cd2be3a8a', 'request_id': 'req-5c130255-ab77-4ff5-82d4-24a440866e82'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:11.325302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5c130255-ab77-4ff5-82d4-24a440866e82 admin admin] The token cache is being invalidate because project bb9f84f2d16a4389a4be567cd2be3a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:29:11.325302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5c130255-ab77-4ff5-82d4-24a440866e82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:11.325302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-5c130255-ab77-4ff5-82d4-24a440866e82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:29:11.325302 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1406/2809] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:29:11 2022] DELETE /identity/v3/projects/bb9f84f2d16a4389a4be567cd2be3a8a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:29:11.327831 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdfe87-386b-4d2f-aa6a-071b10b340e0 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:11.328319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdfe87-386b-4d2f-aa6a-071b10b340e0 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:11.328543 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdfe87-386b-4d2f-aa6a-071b10b340e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:11.348070 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816afb1b-a413-4cee-9b19-308001680175 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:11.348347 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816afb1b-a413-4cee-9b19-308001680175 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:11.348658 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816afb1b-a413-4cee-9b19-308001680175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:11.353301 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-31bdfe87-386b-4d2f-aa6a-071b10b340e0 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:11.354750 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-31bdfe87-386b-4d2f-aa6a-071b10b340e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:11.386526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-816afb1b-a413-4cee-9b19-308001680175 None None] MFA Rules not processed for user `a63e79700c854a3f9581a339ee38376e`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:11.388271 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-816afb1b-a413-4cee-9b19-308001680175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:11.392055 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:29:11.430573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1404/2810] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:29:11.438691 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1daa47-3b25-44e2-8c45-791cf97567d2 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:11.438967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1daa47-3b25-44e2-8c45-791cf97567d2 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:11.439229 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1daa47-3b25-44e2-8c45-791cf97567d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:11.460529 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0a1daa47-3b25-44e2-8c45-791cf97567d2 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:11.461905 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0a1daa47-3b25-44e2-8c45-791cf97567d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:11.470162 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1407/2811] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:29:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:29:11.470952 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1405/2812] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:29:11.476928 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816afb1b-a413-4cee-9b19-308001680175 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:11.481559 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1daa47-3b25-44e2-8c45-791cf97567d2 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:11.482948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0a1daa47-3b25-44e2-8c45-791cf97567d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:11.493035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816afb1b-a413-4cee-9b19-308001680175 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:11.500422 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17653c9-9ee2-415d-aa79-82d8fe88c8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:11.502036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17653c9-9ee2-415d-aa79-82d8fe88c8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:11.502036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17653c9-9ee2-415d-aa79-82d8fe88c8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:11.502036 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17653c9-9ee2-415d-aa79-82d8fe88c8d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:11.502909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17653c9-9ee2-415d-aa79-82d8fe88c8d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81864) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:29:11.503029 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17653c9-9ee2-415d-aa79-82d8fe88c8d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:11.504627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17653c9-9ee2-415d-aa79-82d8fe88c8d4 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:11.523707 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1408/2813] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:29:11.525783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17653c9-9ee2-415d-aa79-82d8fe88c8d4 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:11.536765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17653c9-9ee2-415d-aa79-82d8fe88c8d4 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:11.542161 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df493b-901a-44a9-a34d-52ec72f6dd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:11.542702 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df493b-901a-44a9-a34d-52ec72f6dd07 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:11.542976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df493b-901a-44a9-a34d-52ec72f6dd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:11.543243 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df493b-901a-44a9-a34d-52ec72f6dd07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:11.543943 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df493b-901a-44a9-a34d-52ec72f6dd07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:11.545871 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df493b-901a-44a9-a34d-52ec72f6dd07 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:11.565560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1daa47-3b25-44e2-8c45-791cf97567d2 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:11.569507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b98342-91c3-4d41-9165-7e6dbafe0965 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:11.570196 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b98342-91c3-4d41-9165-7e6dbafe0965 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:11.570552 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b98342-91c3-4d41-9165-7e6dbafe0965 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:11.570759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b98342-91c3-4d41-9165-7e6dbafe0965 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:11.578607 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1409/2814] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:29:11 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:29:11.581705 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b98342-91c3-4d41-9165-7e6dbafe0965 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:11.582521 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df493b-901a-44a9-a34d-52ec72f6dd07 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:11.583154 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b98342-91c3-4d41-9165-7e6dbafe0965 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:11.593239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1406/2815] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:29:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:29:11.633404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:29:11.650103 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df493b-901a-44a9-a34d-52ec72f6dd07 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:11.655484 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65654b8d-b2f5-417c-b5dd-00bb1b9bbda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:11.656042 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65654b8d-b2f5-417c-b5dd-00bb1b9bbda9 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:11.656321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65654b8d-b2f5-417c-b5dd-00bb1b9bbda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:11.656617 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65654b8d-b2f5-417c-b5dd-00bb1b9bbda9 admin admin] PATH_INFO: `/v3/users` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:11.657255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65654b8d-b2f5-417c-b5dd-00bb1b9bbda9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:11.659193 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65654b8d-b2f5-417c-b5dd-00bb1b9bbda9 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:11.681970 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1410/2816] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:29:11 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:29:11.686242 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b98342-91c3-4d41-9165-7e6dbafe0965 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:11.697185 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b98342-91c3-4d41-9165-7e6dbafe0965 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:11.702685 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914828d-926a-4e94-a11d-9ce7053160f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:11.703387 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914828d-926a-4e94-a11d-9ce7053160f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:11.703757 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914828d-926a-4e94-a11d-9ce7053160f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:11.704030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914828d-926a-4e94-a11d-9ce7053160f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:11.704720 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914828d-926a-4e94-a11d-9ce7053160f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:11.706318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914828d-926a-4e94-a11d-9ce7053160f7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:11.717403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1407/2817] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:29:11.722086 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65654b8d-b2f5-417c-b5dd-00bb1b9bbda9 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:11.732282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65654b8d-b2f5-417c-b5dd-00bb1b9bbda9 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:11.737334 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5f718-6277-4192-94f9-a932436d9b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:11.737935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5f718-6277-4192-94f9-a932436d9b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:11.738211 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5f718-6277-4192-94f9-a932436d9b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:11.738480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5f718-6277-4192-94f9-a932436d9b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:11.739133 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5f718-6277-4192-94f9-a932436d9b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:11.741113 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5f718-6277-4192-94f9-a932436d9b6b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:11.752695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1411/2818] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:29:11.757197 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914828d-926a-4e94-a11d-9ce7053160f7 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:11.767780 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914828d-926a-4e94-a11d-9ce7053160f7 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:11.772945 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2497b86-166c-4f6e-8169-495485b5d203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:11.773617 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2497b86-166c-4f6e-8169-495485b5d203 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:11.773884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2497b86-166c-4f6e-8169-495485b5d203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:11.774203 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2497b86-166c-4f6e-8169-495485b5d203 admin admin] PATH_INFO: `/v3/projects/58f43b3aab03420bbcb0a2b0f7a338ac/users/338a34cb85d7455a9bed32158d4f9497/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:11.789811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2497b86-166c-4f6e-8169-495485b5d203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f43b3aab03420bbcb0a2b0f7a338ac, user_id=338a34cb85d7455a9bed32158d4f9497, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:11.792539 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2497b86-166c-4f6e-8169-495485b5d203 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:11.807658 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1408/2819] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:29:11 2022] PUT /identity/v3/projects/58f43b3aab03420bbcb0a2b0f7a338ac/users/338a34cb85d7455a9bed32158d4f9497/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:29:11.813011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5074c-afed-4a3c-9e02-3155b655a1f7 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:11.813296 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5074c-afed-4a3c-9e02-3155b655a1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:11.813598 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5074c-afed-4a3c-9e02-3155b655a1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:11.834599 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-05c5074c-afed-4a3c-9e02-3155b655a1f7 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:11.835796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-05c5074c-afed-4a3c-9e02-3155b655a1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:11.908655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1412/2820] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:29:11.916978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2497b86-166c-4f6e-8169-495485b5d203 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:11.918811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f2497b86-166c-4f6e-8169-495485b5d203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:11.977489 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2497b86-166c-4f6e-8169-495485b5d203 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:11.981180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e920a69-cf3c-4169-aafd-8758804750f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:11.981788 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e920a69-cf3c-4169-aafd-8758804750f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:11.982058 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e920a69-cf3c-4169-aafd-8758804750f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:11.982322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e920a69-cf3c-4169-aafd-8758804750f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:11.991294 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e920a69-cf3c-4169-aafd-8758804750f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:11.992756 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e920a69-cf3c-4169-aafd-8758804750f2 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:12.002575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1409/2821] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:29:13.046854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77655101-1395-4da5-9c7f-0b8b63d86386 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:13.047432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77655101-1395-4da5-9c7f-0b8b63d86386 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:13.047432 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77655101-1395-4da5-9c7f-0b8b63d86386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:13.067786 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-77655101-1395-4da5-9c7f-0b8b63d86386 None None] MFA Rules not processed for user `a63e79700c854a3f9581a339ee38376e`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:13.069502 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-77655101-1395-4da5-9c7f-0b8b63d86386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:13.141345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1413/2822] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:29:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:29:14.065467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e920a69-cf3c-4169-aafd-8758804750f2 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:14.067800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-8e920a69-cf3c-4169-aafd-8758804750f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:14.140288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:29:14.171102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e920a69-cf3c-4169-aafd-8758804750f2 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:14.176220 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d872e2-2b53-4442-a38c-3006781d5f27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:14.176892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d872e2-2b53-4442-a38c-3006781d5f27 service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:14.177163 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d872e2-2b53-4442-a38c-3006781d5f27 service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:14.177482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d872e2-2b53-4442-a38c-3006781d5f27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:14.179247 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-28d872e2-2b53-4442-a38c-3006781d5f27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:14.191801 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d872e2-2b53-4442-a38c-3006781d5f27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:14.193573 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d872e2-2b53-4442-a38c-3006781d5f27 service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:14.205206 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1410/2823] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:29:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:29:16.745556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77655101-1395-4da5-9c7f-0b8b63d86386 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:16.748327 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-77655101-1395-4da5-9c7f-0b8b63d86386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:16.767735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77655101-1395-4da5-9c7f-0b8b63d86386 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:16.779293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda595c-ceb2-4e55-a740-c80776f6dbdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:16.780248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda595c-ceb2-4e55-a740-c80776f6dbdb service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:16.781032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda595c-ceb2-4e55-a740-c80776f6dbdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:16.781522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda595c-ceb2-4e55-a740-c80776f6dbdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:16.784081 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3dda595c-ceb2-4e55-a740-c80776f6dbdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:16.802909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda595c-ceb2-4e55-a740-c80776f6dbdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:16.804870 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda595c-ceb2-4e55-a740-c80776f6dbdb service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:16.820065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1414/2824] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:29:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:29:17.169818 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a5470-0e91-4338-a4a6-91b97be54bd9 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:17.170482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a5470-0e91-4338-a4a6-91b97be54bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:17.170482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a5470-0e91-4338-a4a6-91b97be54bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:17.200953 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-967a5470-0e91-4338-a4a6-91b97be54bd9 None None] MFA Rules not processed for user `338a34cb85d7455a9bed32158d4f9497`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:17.202194 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-967a5470-0e91-4338-a4a6-91b97be54bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:17.286466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1411/2825] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:29:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:29:17.301776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda595c-ceb2-4e55-a740-c80776f6dbdb service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:17.303427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-3dda595c-ceb2-4e55-a740-c80776f6dbdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:17.370770 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:29:17.401346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda595c-ceb2-4e55-a740-c80776f6dbdb service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:17.406582 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a394506-4964-4bc8-9030-6fa5df559997 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:17.407205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a394506-4964-4bc8-9030-6fa5df559997 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:17.407478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a394506-4964-4bc8-9030-6fa5df559997 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:17.407741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a394506-4964-4bc8-9030-6fa5df559997 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:17.418900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a394506-4964-4bc8-9030-6fa5df559997 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:17.420568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a394506-4964-4bc8-9030-6fa5df559997 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:17.437175 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1415/2826] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:29:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:29:19.456065 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772709e-fda1-4aa5-809c-ed84ffeaf7c3 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:19.456967 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772709e-fda1-4aa5-809c-ed84ffeaf7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:19.457255 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772709e-fda1-4aa5-809c-ed84ffeaf7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:19.481557 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-5772709e-fda1-4aa5-809c-ed84ffeaf7c3 None None] MFA Rules not processed for user `338a34cb85d7455a9bed32158d4f9497`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:29:19.482889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5772709e-fda1-4aa5-809c-ed84ffeaf7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:19.491264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1412/2827] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:29:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:29:25.468423 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a394506-4964-4bc8-9030-6fa5df559997 service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:29:25.470282 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2a394506-4964-4bc8-9030-6fa5df559997 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:25.487916 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a394506-4964-4bc8-9030-6fa5df559997 service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:29:25.494050 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0dff8-f22a-4250-affe-a7496980823a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:29:25.494752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0dff8-f22a-4250-affe-a7496980823a service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:29:25.495094 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0dff8-f22a-4250-affe-a7496980823a service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:29:25.495429 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0dff8-f22a-4250-affe-a7496980823a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:29:25.497622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-83c0dff8-f22a-4250-affe-a7496980823a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:29:25.510196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0dff8-f22a-4250-affe-a7496980823a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:29:25.511777 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c0dff8-f22a-4250-affe-a7496980823a service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:29:25.528854 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1416/2828] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:29:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:29:29.475631 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=50 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:29:35.518315 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=14 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:30:21.031332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772709e-fda1-4aa5-809c-ed84ffeaf7c3 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:30:21.061980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772709e-fda1-4aa5-809c-ed84ffeaf7c3 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:30:21.073728 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bbdb5-cf74-432f-9bc4-d99d0ffbeabd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:30:21.074673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bbdb5-cf74-432f-9bc4-d99d0ffbeabd service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:21.075155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bbdb5-cf74-432f-9bc4-d99d0ffbeabd service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:21.075739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bbdb5-cf74-432f-9bc4-d99d0ffbeabd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:21.078244 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d88bbdb5-cf74-432f-9bc4-d99d0ffbeabd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:21.097426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88bbdb5-cf74-432f-9bc4-d99d0ffbeabd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:30:21.099056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88bbdb5-cf74-432f-9bc4-d99d0ffbeabd service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:30:21.120056 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1413/2829] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:30:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:30:31.106258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:30:33.105751 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd947aa-8227-4b54-a396-033b56cee333 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:33.106295 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd947aa-8227-4b54-a396-033b56cee333 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:33.106295 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd947aa-8227-4b54-a396-033b56cee333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:33.127256 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-2dd947aa-8227-4b54-a396-033b56cee333 None None] MFA Rules not processed for user `a63e79700c854a3f9581a339ee38376e`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:30:33.128462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-2dd947aa-8227-4b54-a396-033b56cee333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:33.137316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1417/2830] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:30:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:30:33.387993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bbdb5-cf74-432f-9bc4-d99d0ffbeabd service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:30:33.398485 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bbdb5-cf74-432f-9bc4-d99d0ffbeabd service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:30:33.402679 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64e23d-4f49-40ba-a5df-2179ac0c9007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:30:33.403302 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64e23d-4f49-40ba-a5df-2179ac0c9007 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:33.403576 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64e23d-4f49-40ba-a5df-2179ac0c9007 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:33.403836 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64e23d-4f49-40ba-a5df-2179ac0c9007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:33.405968 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0a64e23d-4f49-40ba-a5df-2179ac0c9007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:33.417215 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64e23d-4f49-40ba-a5df-2179ac0c9007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:30:33.418718 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64e23d-4f49-40ba-a5df-2179ac0c9007 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:30:33.427216 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1414/2831] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:30:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:30:43.120923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:30:43.423245 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:30:46.088464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdf216-e230-4895-ba79-94899bdf4c1d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:46.089047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdf216-e230-4895-ba79-94899bdf4c1d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:46.089047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdf216-e230-4895-ba79-94899bdf4c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:46.111166 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f0fdf216-e230-4895-ba79-94899bdf4c1d None None] MFA Rules not processed for user `cd5b918b0427408d9a70ca568d75ad58`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:30:46.112386 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f0fdf216-e230-4895-ba79-94899bdf4c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:46.174731 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1418/2832] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:30:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:30:46.506273 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64e23d-4f49-40ba-a5df-2179ac0c9007 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:30:46.515341 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64e23d-4f49-40ba-a5df-2179ac0c9007 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:30:46.519507 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8faac-7197-4e61-bce4-fd54d03f40a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:30:46.520098 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8faac-7197-4e61-bce4-fd54d03f40a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:30:46.520373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8faac-7197-4e61-bce4-fd54d03f40a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:30:46.520662 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8faac-7197-4e61-bce4-fd54d03f40a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:30:46.522464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-01a8faac-7197-4e61-bce4-fd54d03f40a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:30:46.532265 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8faac-7197-4e61-bce4-fd54d03f40a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:30:46.533786 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8faac-7197-4e61-bce4-fd54d03f40a2 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:30:46.542720 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1415/2833] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:30:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:30:56.167034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=25 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:30:56.538370 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:31:13.004095 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdf216-e230-4895-ba79-94899bdf4c1d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:31:13.018843 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdf216-e230-4895-ba79-94899bdf4c1d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:31:13.024175 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98c6ef-15f4-4f61-9a90-17efa436f4be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:31:13.024822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98c6ef-15f4-4f61-9a90-17efa436f4be service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:31:13.025091 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98c6ef-15f4-4f61-9a90-17efa436f4be service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:31:13.025353 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98c6ef-15f4-4f61-9a90-17efa436f4be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:31:13.027867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1d98c6ef-15f4-4f61-9a90-17efa436f4be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:31:13.040465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98c6ef-15f4-4f61-9a90-17efa436f4be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:31:13.042261 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98c6ef-15f4-4f61-9a90-17efa436f4be service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:31:13.055758 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1419/2834] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:31:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:31:13.923013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8faac-7197-4e61-bce4-fd54d03f40a2 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:31:13.937358 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8faac-7197-4e61-bce4-fd54d03f40a2 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:31:13.943194 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7811ae1-a163-4e8c-8b3e-ffa5fdc97cbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:31:13.943794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7811ae1-a163-4e8c-8b3e-ffa5fdc97cbe service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:31:13.944063 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7811ae1-a163-4e8c-8b3e-ffa5fdc97cbe service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:31:13.944323 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7811ae1-a163-4e8c-8b3e-ffa5fdc97cbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:31:13.946133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b7811ae1-a163-4e8c-8b3e-ffa5fdc97cbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:31:13.960166 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7811ae1-a163-4e8c-8b3e-ffa5fdc97cbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:31:13.961747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7811ae1-a163-4e8c-8b3e-ffa5fdc97cbe service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:31:13.972991 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1416/2835] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:31:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:31:23.049652 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:31:23.968073 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:31:39.084867 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98c6ef-15f4-4f61-9a90-17efa436f4be service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:31:39.086337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1d98c6ef-15f4-4f61-9a90-17efa436f4be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:31:39.099238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98c6ef-15f4-4f61-9a90-17efa436f4be service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:31:39.104744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbb63e-5b8a-4803-8646-300465dad9a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:31:39.105317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbb63e-5b8a-4803-8646-300465dad9a2 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:31:39.105618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbb63e-5b8a-4803-8646-300465dad9a2 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:31:39.105860 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbb63e-5b8a-4803-8646-300465dad9a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:31:39.107593 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-82bbb63e-5b8a-4803-8646-300465dad9a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:31:39.120279 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbb63e-5b8a-4803-8646-300465dad9a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:31:39.122197 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbb63e-5b8a-4803-8646-300465dad9a2 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:31:39.135623 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1420/2836] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:31:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:31:39.140000 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7811ae1-a163-4e8c-8b3e-ffa5fdc97cbe service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:31:39.142050 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b7811ae1-a163-4e8c-8b3e-ffa5fdc97cbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:31:39.156484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7811ae1-a163-4e8c-8b3e-ffa5fdc97cbe service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:31:39.161430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44760be-d4c1-4d89-9514-ee2cbb2e6781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:31:39.162328 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44760be-d4c1-4d89-9514-ee2cbb2e6781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:31:39.162672 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44760be-d4c1-4d89-9514-ee2cbb2e6781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:31:39.163015 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44760be-d4c1-4d89-9514-ee2cbb2e6781 admin admin] PATH_INFO: `/v3/users/a63e79700c854a3f9581a339ee38376e` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:31:39.164401 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44760be-d4c1-4d89-9514-ee2cbb2e6781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a63e79700c854a3f9581a339ee38376e)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:31:39.167584 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44760be-d4c1-4d89-9514-ee2cbb2e6781 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:31:39.220476 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a44760be-d4c1-4d89-9514-ee2cbb2e6781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a63e79700c854a3f9581a339ee38376e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:31:39.227364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a44760be-d4c1-4d89-9514-ee2cbb2e6781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a63e79700c854a3f9581a339ee38376e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:31:39.242195 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a44760be-d4c1-4d89-9514-ee2cbb2e6781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a63e79700c854a3f9581a339ee38376e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:31:39.251837 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a44760be-d4c1-4d89-9514-ee2cbb2e6781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a63e79700c854a3f9581a339ee38376e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:31:39.252671 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-a44760be-d4c1-4d89-9514-ee2cbb2e6781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a63e79700c854a3f9581a339ee38376e'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:31:39.258882 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1417/2837] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:31:39 2022] DELETE /identity/v3/users/a63e79700c854a3f9581a339ee38376e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:31:39.445674 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbb63e-5b8a-4803-8646-300465dad9a2 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:31:39.461382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbb63e-5b8a-4803-8646-300465dad9a2 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:31:39.521257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:31:39.521893 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:31:39.522168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:31:39.522435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] PATH_INFO: `/v3/projects/ace0c8e4f20649e28610cde631bc58b8` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:31:39.523539 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ace0c8e4f20649e28610cde631bc58b8)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:31:39.525535 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:31:39.574193 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ace0c8e4f20649e28610cde631bc58b8', 'request_id': 'req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:31:39.583676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ace0c8e4f20649e28610cde631bc58b8', 'request_id': 'req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:31:39.590540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ace0c8e4f20649e28610cde631bc58b8', 'request_id': 'req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:31:39.600895 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] The token cache is being invalidate because project ace0c8e4f20649e28610cde631bc58b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:31:39.601151 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:31:39.601837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:31:39.603357 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1421/2838] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:31:39 2022] DELETE /identity/v3/projects/ace0c8e4f20649e28610cde631bc58b8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:31:44.032172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a76c1e-d628-4308-a58e-e4cc26cbbc7e None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:31:44.032698 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a76c1e-d628-4308-a58e-e4cc26cbbc7e None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:31:44.032830 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a76c1e-d628-4308-a58e-e4cc26cbbc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:31:44.054332 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-a1a76c1e-d628-4308-a58e-e4cc26cbbc7e None None] MFA Rules not processed for user `338a34cb85d7455a9bed32158d4f9497`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:31:44.055584 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a1a76c1e-d628-4308-a58e-e4cc26cbbc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:31:44.112938 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1418/2839] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:31:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:31:49.598086 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:31:54.105703 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=54,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:01.241569 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:01.243404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:01.463149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23bbef-e47f-4bd3-9f81-5706daeb24e3 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:01.468894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290773b-ea43-46e1-bb0c-5f461f255f6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:01.469507 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290773b-ea43-46e1-bb0c-5f461f255f6f service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:01.469785 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290773b-ea43-46e1-bb0c-5f461f255f6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:01.470047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290773b-ea43-46e1-bb0c-5f461f255f6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:01.471923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b290773b-ea43-46e1-bb0c-5f461f255f6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:01.552753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290773b-ea43-46e1-bb0c-5f461f255f6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:01.554210 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290773b-ea43-46e1-bb0c-5f461f255f6f service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:01.566417 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1422/2840] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:03.017928 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a76c1e-d628-4308-a58e-e4cc26cbbc7e None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:03.019554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-a1a76c1e-d628-4308-a58e-e4cc26cbbc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:03.081206 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a76c1e-d628-4308-a58e-e4cc26cbbc7e None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:03.085712 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860be2f-10b2-4565-baa4-f6037b1207cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:03.086316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860be2f-10b2-4565-baa4-f6037b1207cb service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:03.086584 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860be2f-10b2-4565-baa4-f6037b1207cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:03.086842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860be2f-10b2-4565-baa4-f6037b1207cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:03.088794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5860be2f-10b2-4565-baa4-f6037b1207cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:03.102963 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860be2f-10b2-4565-baa4-f6037b1207cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:03.104362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860be2f-10b2-4565-baa4-f6037b1207cb service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:03.115506 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1419/2841] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:03.707906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290773b-ea43-46e1-bb0c-5f461f255f6f service cinder] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:03.709352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b290773b-ea43-46e1-bb0c-5f461f255f6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:03.721286 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290773b-ea43-46e1-bb0c-5f461f255f6f service cinder] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:03.725464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b0a00-e945-4e55-b841-cf58a964e2e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:03.726107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b0a00-e945-4e55-b841-cf58a964e2e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:03.726384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b0a00-e945-4e55-b841-cf58a964e2e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:03.726664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b0a00-e945-4e55-b841-cf58a964e2e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:03.728467 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f73b0a00-e945-4e55-b841-cf58a964e2e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:03.789474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b0a00-e945-4e55-b841-cf58a964e2e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:03.790905 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b0a00-e945-4e55-b841-cf58a964e2e3 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:03.801532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1423/2842] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:13.110909 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=26 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:13.795741 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=82 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:14.958232 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee920127-7849-46b0-b0ba-db92bdf193bb None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:14.958776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee920127-7849-46b0-b0ba-db92bdf193bb None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:14.958776 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee920127-7849-46b0-b0ba-db92bdf193bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:14.985114 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ee920127-7849-46b0-b0ba-db92bdf193bb None None] MFA Rules not processed for user `cd5b918b0427408d9a70ca568d75ad58`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:14.987004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ee920127-7849-46b0-b0ba-db92bdf193bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:15.003139 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1420/2843] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:33:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:22.467189 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d2d8a-0e0a-41f8-bd0c-f4444779d922 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:22.468180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d2d8a-0e0a-41f8-bd0c-f4444779d922 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:22.468471 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d2d8a-0e0a-41f8-bd0c-f4444779d922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:22.487557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-f30d2d8a-0e0a-41f8-bd0c-f4444779d922 None None] MFA Rules not processed for user `338a34cb85d7455a9bed32158d4f9497`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:33:22.488717 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f30d2d8a-0e0a-41f8-bd0c-f4444779d922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:22.497110 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1424/2844] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:33:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:33:24.975926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:32.482251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:33:38.037898 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee920127-7849-46b0-b0ba-db92bdf193bb None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:33:38.053772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee920127-7849-46b0-b0ba-db92bdf193bb None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:33:38.060907 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fea38-2e6d-470a-8605-42ab173d50b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:33:38.061715 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fea38-2e6d-470a-8605-42ab173d50b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:33:38.062066 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fea38-2e6d-470a-8605-42ab173d50b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:33:38.062791 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fea38-2e6d-470a-8605-42ab173d50b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:33:38.065204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-df7fea38-2e6d-470a-8605-42ab173d50b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:33:38.137400 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fea38-2e6d-470a-8605-42ab173d50b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:33:38.138830 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fea38-2e6d-470a-8605-42ab173d50b6 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:33:38.149000 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1421/2845] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:33:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:33:48.144487 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=30 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:34:50.436089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d2d8a-0e0a-41f8-bd0c-f4444779d922 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:34:50.437566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-f30d2d8a-0e0a-41f8-bd0c-f4444779d922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:50.453842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d2d8a-0e0a-41f8-bd0c-f4444779d922 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:34:50.460124 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfec89e-284a-403a-b394-e312051cd80a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:34:50.460765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfec89e-284a-403a-b394-e312051cd80a service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:34:50.461039 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfec89e-284a-403a-b394-e312051cd80a service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:34:50.461305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfec89e-284a-403a-b394-e312051cd80a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:34:50.463163 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-abfec89e-284a-403a-b394-e312051cd80a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:34:50.473792 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfec89e-284a-403a-b394-e312051cd80a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:34:50.475356 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfec89e-284a-403a-b394-e312051cd80a service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:34:50.484815 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1425/2846] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:34:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:35:00.480336 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=6 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:35:08.481025 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12752c-1895-4bde-9e15-8e07949f7679 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:35:08.481629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12752c-1895-4bde-9e15-8e07949f7679 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:35:08.481629 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12752c-1895-4bde-9e15-8e07949f7679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:35:08.503890 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-4c12752c-1895-4bde-9e15-8e07949f7679 None None] MFA Rules not processed for user `338a34cb85d7455a9bed32158d4f9497`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:35:08.505426 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-4c12752c-1895-4bde-9e15-8e07949f7679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:35:08.513317 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1422/2847] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:35:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:35:09.179918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfec89e-284a-403a-b394-e312051cd80a service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:35:09.181408 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-abfec89e-284a-403a-b394-e312051cd80a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:35:09.251140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfec89e-284a-403a-b394-e312051cd80a service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:35:09.256276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4909cb-1bec-49e7-af9c-32b7356d69c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:35:09.256918 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4909cb-1bec-49e7-af9c-32b7356d69c0 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:35:09.257194 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4909cb-1bec-49e7-af9c-32b7356d69c0 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:35:09.257461 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4909cb-1bec-49e7-af9c-32b7356d69c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:35:09.259196 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-4d4909cb-1bec-49e7-af9c-32b7356d69c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:35:09.270658 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4909cb-1bec-49e7-af9c-32b7356d69c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:35:09.272233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4909cb-1bec-49e7-af9c-32b7356d69c0 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:35:09.282144 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1426/2848] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:35:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:35:11.323465 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12752c-1895-4bde-9e15-8e07949f7679 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:35:11.334285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12752c-1895-4bde-9e15-8e07949f7679 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:35:11.339393 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc344ac5-c7f5-4e25-b964-2246d17c4bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:35:11.340308 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc344ac5-c7f5-4e25-b964-2246d17c4bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:35:11.340579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc344ac5-c7f5-4e25-b964-2246d17c4bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:35:11.340964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc344ac5-c7f5-4e25-b964-2246d17c4bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:35:11.342748 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fc344ac5-c7f5-4e25-b964-2246d17c4bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:35:11.353970 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc344ac5-c7f5-4e25-b964-2246d17c4bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:35:11.355563 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc344ac5-c7f5-4e25-b964-2246d17c4bd5 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:35:11.365319 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1423/2849] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:35:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:35:16.703348 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4909cb-1bec-49e7-af9c-32b7356d69c0 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:35:16.716745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4909cb-1bec-49e7-af9c-32b7356d69c0 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:35:16.726129 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320eaa45-da54-44cc-ad0b-03f66745c2fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:35:16.726651 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320eaa45-da54-44cc-ad0b-03f66745c2fc service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:35:16.726919 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320eaa45-da54-44cc-ad0b-03f66745c2fc service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:35:16.727180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320eaa45-da54-44cc-ad0b-03f66745c2fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:35:16.729230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-320eaa45-da54-44cc-ad0b-03f66745c2fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:35:16.743110 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320eaa45-da54-44cc-ad0b-03f66745c2fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:35:16.744760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320eaa45-da54-44cc-ad0b-03f66745c2fc service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:35:16.755891 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1427/2850] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:35:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:35:21.360976 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=9 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:35:21.436892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc344ac5-c7f5-4e25-b964-2246d17c4bd5 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:35:21.440084 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-fc344ac5-c7f5-4e25-b964-2246d17c4bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:35:21.457355 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc344ac5-c7f5-4e25-b964-2246d17c4bd5 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:35:21.463418 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cec20f-70cd-4530-9f43-7130b0fbd69a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:35:21.464036 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cec20f-70cd-4530-9f43-7130b0fbd69a service nova] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:35:21.464308 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cec20f-70cd-4530-9f43-7130b0fbd69a service nova] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:35:21.464572 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cec20f-70cd-4530-9f43-7130b0fbd69a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:35:21.466560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-98cec20f-70cd-4530-9f43-7130b0fbd69a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:35:21.480505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cec20f-70cd-4530-9f43-7130b0fbd69a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:35:21.482116 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cec20f-70cd-4530-9f43-7130b0fbd69a service nova] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:35:21.493779 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1424/2851] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:35:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:35:26.751250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=38 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:35:31.488995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:35:42.606264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320eaa45-da54-44cc-ad0b-03f66745c2fc service nova] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:35:42.616425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320eaa45-da54-44cc-ad0b-03f66745c2fc service nova] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:35:42.620265 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970fd6f0-f474-4d45-a8d7-54ca32e27720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:35:42.621479 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fd6f0-f474-4d45-a8d7-54ca32e27720 service neutron] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:35:42.621753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fd6f0-f474-4d45-a8d7-54ca32e27720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:35:42.622043 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970fd6f0-f474-4d45-a8d7-54ca32e27720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:35:42.623920 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-970fd6f0-f474-4d45-a8d7-54ca32e27720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:35:42.688864 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970fd6f0-f474-4d45-a8d7-54ca32e27720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:35:42.690331 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970fd6f0-f474-4d45-a8d7-54ca32e27720 service neutron] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:35:42.700032 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1428/2852] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:35:45.598226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cec20f-70cd-4530-9f43-7130b0fbd69a service nova] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:35:45.614326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cec20f-70cd-4530-9f43-7130b0fbd69a service nova] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:35:45.629445 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9cfe4-e207-41fc-b2b2-eb06cb49b825 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:35:45.630254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9cfe4-e207-41fc-b2b2-eb06cb49b825 service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:35:45.630625 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9cfe4-e207-41fc-b2b2-eb06cb49b825 service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:35:45.630986 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9cfe4-e207-41fc-b2b2-eb06cb49b825 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:35:45.633035 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d3f9cfe4-e207-41fc-b2b2-eb06cb49b825 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:35:45.645739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9cfe4-e207-41fc-b2b2-eb06cb49b825 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:35:45.647648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9cfe4-e207-41fc-b2b2-eb06cb49b825 service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:35:45.662642 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1425/2853] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:35:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:35:48.906676 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970fd6f0-f474-4d45-a8d7-54ca32e27720 service neutron] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:35:48.910213 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-970fd6f0-f474-4d45-a8d7-54ca32e27720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:35:48.924168 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970fd6f0-f474-4d45-a8d7-54ca32e27720 service neutron] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:35:48.929053 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:35:48.929748 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:35:48.930016 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:35:48.930464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] PATH_INFO: `/v3/users/cd5b918b0427408d9a70ca568d75ad58` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:35:48.931814 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd5b918b0427408d9a70ca568d75ad58)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:35:48.934271 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:35:48.995714 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd5b918b0427408d9a70ca568d75ad58'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:48.996678 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd5b918b0427408d9a70ca568d75ad58'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.006369 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd5b918b0427408d9a70ca568d75ad58'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.018780 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd5b918b0427408d9a70ca568d75ad58'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.019599 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd5b918b0427408d9a70ca568d75ad58'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.038540 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1429/2854] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:35:48 2022] DELETE /identity/v3/users/cd5b918b0427408d9a70ca568d75ad58 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:35:49.231124 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9cfe4-e207-41fc-b2b2-eb06cb49b825 service placement] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:35:49.243464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9cfe4-e207-41fc-b2b2-eb06cb49b825 service placement] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:35:49.306182 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:35:49.306805 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:35:49.307078 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:35:49.307334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] PATH_INFO: `/v3/projects/c05aa308acb842198935488c659d9ab3` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:35:49.308842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c05aa308acb842198935488c659d9ab3)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:35:49.310733 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:35:49.358630 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c05aa308acb842198935488c659d9ab3', 'request_id': 'req-55c46f23-e044-4c29-9aaf-b42a1f79fab1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.364842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c05aa308acb842198935488c659d9ab3', 'request_id': 'req-55c46f23-e044-4c29-9aaf-b42a1f79fab1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.372006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c05aa308acb842198935488c659d9ab3', 'request_id': 'req-55c46f23-e044-4c29-9aaf-b42a1f79fab1'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.389451 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] The token cache is being invalidate because project c05aa308acb842198935488c659d9ab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:35:49.389762 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.390437 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.392003 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1426/2855] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:35:49 2022] DELETE /identity/v3/projects/c05aa308acb842198935488c659d9ab3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:35:49.396106 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:35:49.397634 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:35:49.411474 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e41ad-06bf-4bb7-99db-29f8fe56ce63 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:35:49.470912 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:35:49.471554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:35:49.471822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:35:49.472083 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] PATH_INFO: `/v3/users/cf3bbbbd7285451eb4cacc4e3713bf7a` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:35:49.473246 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf3bbbbd7285451eb4cacc4e3713bf7a)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:35:49.475142 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:35:49.480533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:35:49.528094 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf3bbbbd7285451eb4cacc4e3713bf7a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.528768 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf3bbbbd7285451eb4cacc4e3713bf7a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.535873 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf3bbbbd7285451eb4cacc4e3713bf7a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.548935 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf3bbbbd7285451eb4cacc4e3713bf7a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.549671 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf3bbbbd7285451eb4cacc4e3713bf7a'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.565894 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1430/2856] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:35:49 2022] DELETE /identity/v3/users/cf3bbbbd7285451eb4cacc4e3713bf7a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:35:49.753892 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:35:49.755382 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:35:49.768648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c46f23-e044-4c29-9aaf-b42a1f79fab1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:35:49.828068 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:35:49.828865 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:35:49.829137 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:35:49.829406 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] PATH_INFO: `/v3/projects/d47034a29e454f229fd8f2b851fbd546` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:35:49.830571 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d47034a29e454f229fd8f2b851fbd546)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:35:49.832470 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:35:49.853678 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:35:49.882554 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd47034a29e454f229fd8f2b851fbd546', 'request_id': 'req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.888761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd47034a29e454f229fd8f2b851fbd546', 'request_id': 'req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.894349 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd47034a29e454f229fd8f2b851fbd546', 'request_id': 'req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.908738 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] The token cache is being invalidate because project d47034a29e454f229fd8f2b851fbd546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81865) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:35:49.908995 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.909694 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:35:49.911250 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1427/2857] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:35:49 2022] DELETE /identity/v3/projects/d47034a29e454f229fd8f2b851fbd546 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:35:59.562563 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:35:59.905782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:36:25.834050 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:25.835531 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:26.001836 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df961073-f053-4219-996f-19e03c1999a6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:26.072206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97289f-7812-4cbb-a29c-6ec03ee208a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:26.072856 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97289f-7812-4cbb-a29c-6ec03ee208a4 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:26.073129 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97289f-7812-4cbb-a29c-6ec03ee208a4 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:26.073382 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97289f-7812-4cbb-a29c-6ec03ee208a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:26.075335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1b97289f-7812-4cbb-a29c-6ec03ee208a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:26.086738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97289f-7812-4cbb-a29c-6ec03ee208a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:26.088656 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97289f-7812-4cbb-a29c-6ec03ee208a4 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:26.098551 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1431/2858] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:36:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:36:36.094465 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=89 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:36:45.121575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:45.123100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:45.216950 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40308dc-d27b-43d7-8cb4-4bfafe9c088f admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:45.222409 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee204c70-e8b9-42a3-b250-13b02d48b7e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:45.223322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee204c70-e8b9-42a3-b250-13b02d48b7e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:45.223794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee204c70-e8b9-42a3-b250-13b02d48b7e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:45.224276 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee204c70-e8b9-42a3-b250-13b02d48b7e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:45.227260 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ee204c70-e8b9-42a3-b250-13b02d48b7e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:45.319835 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee204c70-e8b9-42a3-b250-13b02d48b7e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:45.321264 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee204c70-e8b9-42a3-b250-13b02d48b7e2 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:45.339374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1428/2859] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:36:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:36:48.639272 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97289f-7812-4cbb-a29c-6ec03ee208a4 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:48.641005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-1b97289f-7812-4cbb-a29c-6ec03ee208a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:48.655065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97289f-7812-4cbb-a29c-6ec03ee208a4 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:48.661205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d62a6c-a8cb-4e99-9e57-3633adc3d851 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:48.662109 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d62a6c-a8cb-4e99-9e57-3633adc3d851 service placement] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:48.662364 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d62a6c-a8cb-4e99-9e57-3633adc3d851 service placement] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:48.662657 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d62a6c-a8cb-4e99-9e57-3633adc3d851 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:48.664749 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b6d62a6c-a8cb-4e99-9e57-3633adc3d851 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:48.678825 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d62a6c-a8cb-4e99-9e57-3633adc3d851 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:48.681251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d62a6c-a8cb-4e99-9e57-3633adc3d851 service placement] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:48.710073 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1432/2860] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:36:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:36:48.721080 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee204c70-e8b9-42a3-b250-13b02d48b7e2 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:48.723258 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ee204c70-e8b9-42a3-b250-13b02d48b7e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:48.746716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee204c70-e8b9-42a3-b250-13b02d48b7e2 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:48.751373 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c26ae-e3fd-4ea0-9a4b-38f981e8a947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:48.752152 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c26ae-e3fd-4ea0-9a4b-38f981e8a947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:48.752430 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c26ae-e3fd-4ea0-9a4b-38f981e8a947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:48.752771 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c26ae-e3fd-4ea0-9a4b-38f981e8a947 admin admin] PATH_INFO: `/v3/users/338a34cb85d7455a9bed32158d4f9497` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:48.753888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c26ae-e3fd-4ea0-9a4b-38f981e8a947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=338a34cb85d7455a9bed32158d4f9497)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:48.755821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c26ae-e3fd-4ea0-9a4b-38f981e8a947 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:48.804331 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b15c26ae-e3fd-4ea0-9a4b-38f981e8a947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '338a34cb85d7455a9bed32158d4f9497'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:48.808704 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:36:48.812004 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b15c26ae-e3fd-4ea0-9a4b-38f981e8a947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '338a34cb85d7455a9bed32158d4f9497'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:48.825867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b15c26ae-e3fd-4ea0-9a4b-38f981e8a947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '338a34cb85d7455a9bed32158d4f9497'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:48.834435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b15c26ae-e3fd-4ea0-9a4b-38f981e8a947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '338a34cb85d7455a9bed32158d4f9497'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:48.835086 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-b15c26ae-e3fd-4ea0-9a4b-38f981e8a947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '338a34cb85d7455a9bed32158d4f9497'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:48.841069 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1429/2861] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:36:48 2022] DELETE /identity/v3/users/338a34cb85d7455a9bed32158d4f9497 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:36:49.026524 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d62a6c-a8cb-4e99-9e57-3633adc3d851 service placement] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:49.038075 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d62a6c-a8cb-4e99-9e57-3633adc3d851 service placement] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:49.096329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:49.097212 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:49.097630 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:49.098116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0 admin admin] PATH_INFO: `/v3/projects/58f43b3aab03420bbcb0a2b0f7a338ac` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:49.099387 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58f43b3aab03420bbcb0a2b0f7a338ac)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:49.101726 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:49.158206 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58f43b3aab03420bbcb0a2b0f7a338ac', 'request_id': 'req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:49.167861 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58f43b3aab03420bbcb0a2b0f7a338ac', 'request_id': 'req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:49.174464 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58f43b3aab03420bbcb0a2b0f7a338ac', 'request_id': 'req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:49.186317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0 admin admin] The token cache is being invalidate because project 58f43b3aab03420bbcb0a2b0f7a338ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:36:49.186602 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:49.187299 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0c34cba7-8ede-4dc2-830a-ed2282c4dbf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:49.188974 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1433/2862] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:36:49 2022] DELETE /identity/v3/projects/58f43b3aab03420bbcb0a2b0f7a338ac => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:36:49.194308 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd7f4f-4090-4ace-90bb-e38b4d1e9ce8 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:49.194533 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd7f4f-4090-4ace-90bb-e38b4d1e9ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:49.194804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd7f4f-4090-4ace-90bb-e38b4d1e9ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:49.258099 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-39fd7f4f-4090-4ace-90bb-e38b4d1e9ce8 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:36:49.259322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-39fd7f4f-4090-4ace-90bb-e38b4d1e9ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:49.322106 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1430/2863] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:36:49.330242 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56509d5d-97cb-4a28-ac3a-24020fc9bc0c None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:49.330526 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56509d5d-97cb-4a28-ac3a-24020fc9bc0c None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:49.330796 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56509d5d-97cb-4a28-ac3a-24020fc9bc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:49.352205 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-56509d5d-97cb-4a28-ac3a-24020fc9bc0c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:36:49.353445 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-56509d5d-97cb-4a28-ac3a-24020fc9bc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:49.363499 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1434/2864] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:36:49.368235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd7f4f-4090-4ace-90bb-e38b4d1e9ce8 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:49.378320 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd7f4f-4090-4ace-90bb-e38b4d1e9ce8 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:49.384794 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49617d-5f82-4cff-b19f-4a6f59a05d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:49.385395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49617d-5f82-4cff-b19f-4a6f59a05d2d admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:49.385666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49617d-5f82-4cff-b19f-4a6f59a05d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:49.385912 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49617d-5f82-4cff-b19f-4a6f59a05d2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:49.386560 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49617d-5f82-4cff-b19f-4a6f59a05d2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:36:49.386858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49617d-5f82-4cff-b19f-4a6f59a05d2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:49.388444 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49617d-5f82-4cff-b19f-4a6f59a05d2d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:49.400827 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1431/2865] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:36:49.405150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56509d5d-97cb-4a28-ac3a-24020fc9bc0c None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:49.414543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56509d5d-97cb-4a28-ac3a-24020fc9bc0c None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:49.420035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6e91e-036c-41c3-b41e-76698f768922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:49.420633 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6e91e-036c-41c3-b41e-76698f768922 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:49.420938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6e91e-036c-41c3-b41e-76698f768922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:49.421188 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6e91e-036c-41c3-b41e-76698f768922 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:49.421865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6e91e-036c-41c3-b41e-76698f768922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:49.423795 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6e91e-036c-41c3-b41e-76698f768922 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:49.455458 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1435/2866] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:36:49 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:36:49.460398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49617d-5f82-4cff-b19f-4a6f59a05d2d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:49.470440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49617d-5f82-4cff-b19f-4a6f59a05d2d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:49.531467 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6321f-948e-4ccb-97b3-dc96346faae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:49.532066 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6321f-948e-4ccb-97b3-dc96346faae1 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:49.532334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6321f-948e-4ccb-97b3-dc96346faae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:49.532607 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6321f-948e-4ccb-97b3-dc96346faae1 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:49.533239 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6321f-948e-4ccb-97b3-dc96346faae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:49.535283 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6321f-948e-4ccb-97b3-dc96346faae1 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:49.558486 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1432/2867] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:36:49 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:36:49.563027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6e91e-036c-41c3-b41e-76698f768922 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:49.576115 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6e91e-036c-41c3-b41e-76698f768922 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:49.583380 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba45e2-54df-4dca-b217-7334eaf4fe58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:49.584008 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba45e2-54df-4dca-b217-7334eaf4fe58 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:49.584262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba45e2-54df-4dca-b217-7334eaf4fe58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:49.584532 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba45e2-54df-4dca-b217-7334eaf4fe58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:49.585543 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba45e2-54df-4dca-b217-7334eaf4fe58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:49.587132 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba45e2-54df-4dca-b217-7334eaf4fe58 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:49.598699 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1436/2868] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:36:49.602996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6321f-948e-4ccb-97b3-dc96346faae1 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:49.614155 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6321f-948e-4ccb-97b3-dc96346faae1 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:49.620477 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce6de7-2f4c-4d9d-aee7-e0ebefb93fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:49.621149 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce6de7-2f4c-4d9d-aee7-e0ebefb93fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:49.621424 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce6de7-2f4c-4d9d-aee7-e0ebefb93fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:49.621687 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce6de7-2f4c-4d9d-aee7-e0ebefb93fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:49.622335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce6de7-2f4c-4d9d-aee7-e0ebefb93fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:49.623972 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce6de7-2f4c-4d9d-aee7-e0ebefb93fa5 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:49.635512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1433/2869] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:36:49.643473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba45e2-54df-4dca-b217-7334eaf4fe58 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:49.658232 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba45e2-54df-4dca-b217-7334eaf4fe58 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:49.665648 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d692815-e9e3-4b1e-8ee4-181644964445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:49.666620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d692815-e9e3-4b1e-8ee4-181644964445 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:49.667003 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d692815-e9e3-4b1e-8ee4-181644964445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:49.667350 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d692815-e9e3-4b1e-8ee4-181644964445 admin admin] PATH_INFO: `/v3/projects/c87a7d5b3d1e4caa991fd9e89a5a2083/users/4d8caf7c512f486b85a50261c7cac17f/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:49.688288 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d692815-e9e3-4b1e-8ee4-181644964445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c87a7d5b3d1e4caa991fd9e89a5a2083, user_id=4d8caf7c512f486b85a50261c7cac17f, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:49.691098 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d692815-e9e3-4b1e-8ee4-181644964445 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:49.711105 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1437/2870] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:36:49 2022] PUT /identity/v3/projects/c87a7d5b3d1e4caa991fd9e89a5a2083/users/4d8caf7c512f486b85a50261c7cac17f/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:36:49.725964 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ac03b-f2b2-4973-a029-0a27be2643ce None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:49.726516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ac03b-f2b2-4973-a029-0a27be2643ce None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:49.726516 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ac03b-f2b2-4973-a029-0a27be2643ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:49.734957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:36:49.757884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-f94ac03b-f2b2-4973-a029-0a27be2643ce None None] MFA Rules not processed for user `4d8caf7c512f486b85a50261c7cac17f`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:36:49.759113 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-f94ac03b-f2b2-4973-a029-0a27be2643ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:49.827599 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1434/2871] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:36:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:36:49.839351 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d692815-e9e3-4b1e-8ee4-181644964445 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:49.841294 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-5d692815-e9e3-4b1e-8ee4-181644964445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:49.860112 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:36:49.946280 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d692815-e9e3-4b1e-8ee4-181644964445 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:49.952107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022fef86-452b-41b3-bfeb-3c9e81b837c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2af1ba390fff41a68ff1a0035901c318', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:49.952765 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fef86-452b-41b3-bfeb-3c9e81b837c5 service nova] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:49.953056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fef86-452b-41b3-bfeb-3c9e81b837c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:49.953316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fef86-452b-41b3-bfeb-3c9e81b837c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:49.962473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022fef86-452b-41b3-bfeb-3c9e81b837c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:49.963984 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022fef86-452b-41b3-bfeb-3c9e81b837c5 service nova] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:49.974026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1438/2872] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:36:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:36:58.445777 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ac03b-f2b2-4973-a029-0a27be2643ce None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:58.523876 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ac03b-f2b2-4973-a029-0a27be2643ce None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:58.528888 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:58.529504 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:58.529778 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:58.530048 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] PATH_INFO: `/v3/users/4d8caf7c512f486b85a50261c7cac17f` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:58.531243 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d8caf7c512f486b85a50261c7cac17f)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:58.533207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:58.588293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d8caf7c512f486b85a50261c7cac17f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:58.595766 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d8caf7c512f486b85a50261c7cac17f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:58.610549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d8caf7c512f486b85a50261c7cac17f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:58.620680 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d8caf7c512f486b85a50261c7cac17f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:58.621344 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d8caf7c512f486b85a50261c7cac17f'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:58.627842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1435/2873] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:36:58 2022] DELETE /identity/v3/users/4d8caf7c512f486b85a50261c7cac17f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:36:58.632763 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e47951-1791-49f5-b63e-acd4598c2ff6 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:58.633316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e47951-1791-49f5-b63e-acd4598c2ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:58.633505 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e47951-1791-49f5-b63e-acd4598c2ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:58.653889 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-a4e47951-1791-49f5-b63e-acd4598c2ff6 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:36:58.655101 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a4e47951-1791-49f5-b63e-acd4598c2ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:58.720026 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1439/2874] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:36:58.731111 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:58.732943 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:58.749380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:36:58.798284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a8e44-7ae5-4b84-94df-da92a69737d3 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:58.804519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a6a11-831c-4d3d-bac5-7d062cca48ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:58.805954 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a6a11-831c-4d3d-bac5-7d062cca48ab service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:58.806399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a6a11-831c-4d3d-bac5-7d062cca48ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:58.806858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a6a11-831c-4d3d-bac5-7d062cca48ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:58.820859 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a6a11-831c-4d3d-bac5-7d062cca48ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:58.822288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a6a11-831c-4d3d-bac5-7d062cca48ab service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:58.832836 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1436/2875] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:36:59.109783 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e47951-1791-49f5-b63e-acd4598c2ff6 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:59.111556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-a4e47951-1791-49f5-b63e-acd4598c2ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:59.126892 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e47951-1791-49f5-b63e-acd4598c2ff6 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:59.131735 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1ff1c-da27-4206-8559-2a807b0787da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:59.132544 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1ff1c-da27-4206-8559-2a807b0787da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:59.132964 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1ff1c-da27-4206-8559-2a807b0787da admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:59.133264 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1ff1c-da27-4206-8559-2a807b0787da admin admin] PATH_INFO: `/v3/projects/c87a7d5b3d1e4caa991fd9e89a5a2083` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:59.134401 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1ff1c-da27-4206-8559-2a807b0787da admin admin] RBAC: Authorizing `identity:delete_project(project_id=c87a7d5b3d1e4caa991fd9e89a5a2083)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:59.136250 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1ff1c-da27-4206-8559-2a807b0787da admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:59.187000 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b4c1ff1c-da27-4206-8559-2a807b0787da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c87a7d5b3d1e4caa991fd9e89a5a2083', 'request_id': 'req-b4c1ff1c-da27-4206-8559-2a807b0787da'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:59.196537 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b4c1ff1c-da27-4206-8559-2a807b0787da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c87a7d5b3d1e4caa991fd9e89a5a2083', 'request_id': 'req-b4c1ff1c-da27-4206-8559-2a807b0787da'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:59.205055 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b4c1ff1c-da27-4206-8559-2a807b0787da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c87a7d5b3d1e4caa991fd9e89a5a2083', 'request_id': 'req-b4c1ff1c-da27-4206-8559-2a807b0787da'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:59.212325 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:36:59.221761 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b4c1ff1c-da27-4206-8559-2a807b0787da admin admin] The token cache is being invalidate because project c87a7d5b3d1e4caa991fd9e89a5a2083 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:36:59.222194 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b4c1ff1c-da27-4206-8559-2a807b0787da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:59.223614 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b4c1ff1c-da27-4206-8559-2a807b0787da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:36:59.226480 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1440/2876] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:36:59 2022] DELETE /identity/v3/projects/c87a7d5b3d1e4caa991fd9e89a5a2083 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:36:59.234569 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73f540-74c3-4aee-9ff9-04122575f09c None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:59.234959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73f540-74c3-4aee-9ff9-04122575f09c None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:59.235336 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73f540-74c3-4aee-9ff9-04122575f09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:59.258441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-ae73f540-74c3-4aee-9ff9-04122575f09c None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:36:59.259632 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-ae73f540-74c3-4aee-9ff9-04122575f09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:59.322998 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1437/2877] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:36:59.331954 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86a18f-d545-4403-bcd1-42d62d11ea2d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:59.332309 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86a18f-d545-4403-bcd1-42d62d11ea2d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:59.332675 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86a18f-d545-4403-bcd1-42d62d11ea2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:59.354415 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-fd86a18f-d545-4403-bcd1-42d62d11ea2d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:36:59.355645 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-fd86a18f-d545-4403-bcd1-42d62d11ea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:59.364664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1441/2878] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:36:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:36:59.369818 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73f540-74c3-4aee-9ff9-04122575f09c None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:59.379916 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae73f540-74c3-4aee-9ff9-04122575f09c None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:59.387241 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be8ee0-8224-4b60-847c-2f672176b852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:59.388110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be8ee0-8224-4b60-847c-2f672176b852 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:59.388468 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be8ee0-8224-4b60-847c-2f672176b852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:59.388764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be8ee0-8224-4b60-847c-2f672176b852 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:59.389483 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be8ee0-8224-4b60-847c-2f672176b852 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:36:59.389901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be8ee0-8224-4b60-847c-2f672176b852 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:59.392413 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be8ee0-8224-4b60-847c-2f672176b852 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:59.410175 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1438/2879] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:36:59.415410 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86a18f-d545-4403-bcd1-42d62d11ea2d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:59.425481 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86a18f-d545-4403-bcd1-42d62d11ea2d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:59.430946 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7c9d8-9388-4274-a91a-090a7351224e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:59.431546 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c9d8-9388-4274-a91a-090a7351224e admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:59.431819 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c9d8-9388-4274-a91a-090a7351224e admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:59.432107 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7c9d8-9388-4274-a91a-090a7351224e admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:59.432782 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7c9d8-9388-4274-a91a-090a7351224e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:59.434881 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7c9d8-9388-4274-a91a-090a7351224e admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:59.468367 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1442/2880] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:36:59 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:36:59.472603 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be8ee0-8224-4b60-847c-2f672176b852 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:59.484342 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be8ee0-8224-4b60-847c-2f672176b852 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:59.545749 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25dacd-3b6a-40c3-a57f-443a35e4dd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:59.546365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25dacd-3b6a-40c3-a57f-443a35e4dd3c admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:59.546764 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25dacd-3b6a-40c3-a57f-443a35e4dd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:59.547037 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25dacd-3b6a-40c3-a57f-443a35e4dd3c admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:59.547877 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25dacd-3b6a-40c3-a57f-443a35e4dd3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:59.549698 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25dacd-3b6a-40c3-a57f-443a35e4dd3c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:59.574948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1439/2881] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:36:59 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:36:59.579225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7c9d8-9388-4274-a91a-090a7351224e admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:59.590271 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7c9d8-9388-4274-a91a-090a7351224e admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:59.595440 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61a4d5-56ea-4e33-b16c-99c0a8688f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:59.596027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61a4d5-56ea-4e33-b16c-99c0a8688f07 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:59.596298 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61a4d5-56ea-4e33-b16c-99c0a8688f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:59.596563 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61a4d5-56ea-4e33-b16c-99c0a8688f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:59.597218 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61a4d5-56ea-4e33-b16c-99c0a8688f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:59.598900 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61a4d5-56ea-4e33-b16c-99c0a8688f07 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:59.610837 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1443/2882] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:36:59.615208 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25dacd-3b6a-40c3-a57f-443a35e4dd3c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:59.626598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25dacd-3b6a-40c3-a57f-443a35e4dd3c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:59.632029 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da384bd-0217-4db5-a76d-5f665d395bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:59.632628 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da384bd-0217-4db5-a76d-5f665d395bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:59.632915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da384bd-0217-4db5-a76d-5f665d395bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:59.633180 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da384bd-0217-4db5-a76d-5f665d395bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:59.633817 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da384bd-0217-4db5-a76d-5f665d395bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:59.635759 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da384bd-0217-4db5-a76d-5f665d395bc6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:59.647601 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1440/2883] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:36:59.652100 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61a4d5-56ea-4e33-b16c-99c0a8688f07 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:59.666753 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61a4d5-56ea-4e33-b16c-99c0a8688f07 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:59.672937 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c453ba2-999f-4d4d-b768-79db450995a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:59.674268 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c453ba2-999f-4d4d-b768-79db450995a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:59.674608 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c453ba2-999f-4d4d-b768-79db450995a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:59.674958 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c453ba2-999f-4d4d-b768-79db450995a1 admin admin] PATH_INFO: `/v3/projects/0b094446c5cd402f899c76a8b83a0034/users/0aa6ff5413cb49b780ce6dddd6d487eb/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:59.698910 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c453ba2-999f-4d4d-b768-79db450995a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b094446c5cd402f899c76a8b83a0034, user_id=0aa6ff5413cb49b780ce6dddd6d487eb, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:59.701596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c453ba2-999f-4d4d-b768-79db450995a1 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:59.719760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1444/2884] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:36:59 2022] PUT /identity/v3/projects/0b094446c5cd402f899c76a8b83a0034/users/0aa6ff5413cb49b780ce6dddd6d487eb/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:36:59.736856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f97e33-7524-4789-8f61-f8e101cb8d24 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:59.737117 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f97e33-7524-4789-8f61-f8e101cb8d24 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:59.737362 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f97e33-7524-4789-8f61-f8e101cb8d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:59.778773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-d0f97e33-7524-4789-8f61-f8e101cb8d24 None None] MFA Rules not processed for user `0aa6ff5413cb49b780ce6dddd6d487eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:36:59.780288 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-d0f97e33-7524-4789-8f61-f8e101cb8d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:36:59.792887 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:36:59.878014 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1441/2885] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:36:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:36:59.882947 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c453ba2-999f-4d4d-b768-79db450995a1 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:36:59.951403 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c453ba2-999f-4d4d-b768-79db450995a1 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:36:59.956706 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc6c69-7c48-41b1-b707-95108083f1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:36:59.957317 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc6c69-7c48-41b1-b707-95108083f1aa admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:36:59.957586 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc6c69-7c48-41b1-b707-95108083f1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:36:59.957868 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc6c69-7c48-41b1-b707-95108083f1aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:36:59.958502 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc6c69-7c48-41b1-b707-95108083f1aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:36:59.960541 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc6c69-7c48-41b1-b707-95108083f1aa admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:36:59.991140 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1445/2886] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:36:59 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:36:59.995502 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f97e33-7524-4789-8f61-f8e101cb8d24 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:00.006980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f97e33-7524-4789-8f61-f8e101cb8d24 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:00.071059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f63c7-e75a-4834-9a86-8cdbc76e8112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:00.071648 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f63c7-e75a-4834-9a86-8cdbc76e8112 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:00.071921 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f63c7-e75a-4834-9a86-8cdbc76e8112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:00.072186 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f63c7-e75a-4834-9a86-8cdbc76e8112 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:00.072846 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f63c7-e75a-4834-9a86-8cdbc76e8112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:00.074761 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f63c7-e75a-4834-9a86-8cdbc76e8112 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:00.099775 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1442/2887] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:36:59 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:37:00.103680 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc6c69-7c48-41b1-b707-95108083f1aa admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:00.114321 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc6c69-7c48-41b1-b707-95108083f1aa admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:00.119615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb35c5-3a4e-4c9f-9363-8364d45e60e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:00.120202 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb35c5-3a4e-4c9f-9363-8364d45e60e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:00.120514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb35c5-3a4e-4c9f-9363-8364d45e60e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:00.121113 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb35c5-3a4e-4c9f-9363-8364d45e60e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:00.121734 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb35c5-3a4e-4c9f-9363-8364d45e60e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:00.123337 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb35c5-3a4e-4c9f-9363-8364d45e60e8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:00.134443 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1446/2888] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:37:00.140428 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f63c7-e75a-4834-9a86-8cdbc76e8112 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:00.151221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f63c7-e75a-4834-9a86-8cdbc76e8112 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:00.156716 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9d80b-0799-4e29-a49c-b367670f3f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:00.157364 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9d80b-0799-4e29-a49c-b367670f3f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:00.157634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9d80b-0799-4e29-a49c-b367670f3f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:00.157901 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9d80b-0799-4e29-a49c-b367670f3f1d admin admin] PATH_INFO: `/v3/projects/aeffceb9fb554af5b59e986e65b76bdc/users/a6211b7de3ec49c4abc82553233d4d74/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:00.174589 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9d80b-0799-4e29-a49c-b367670f3f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeffceb9fb554af5b59e986e65b76bdc, user_id=a6211b7de3ec49c4abc82553233d4d74, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:00.177246 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9d80b-0799-4e29-a49c-b367670f3f1d admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:00.192804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1443/2889] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:37:00 2022] PUT /identity/v3/projects/aeffceb9fb554af5b59e986e65b76bdc/users/a6211b7de3ec49c4abc82553233d4d74/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:37:00.197640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb35c5-3a4e-4c9f-9363-8364d45e60e8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:00.213345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb35c5-3a4e-4c9f-9363-8364d45e60e8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:00.276816 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbfaef-b135-4d2c-9a9d-dc1aba94310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:00.277329 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbfaef-b135-4d2c-9a9d-dc1aba94310a admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:00.277736 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbfaef-b135-4d2c-9a9d-dc1aba94310a admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:00.278011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbfaef-b135-4d2c-9a9d-dc1aba94310a admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:00.278660 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbfaef-b135-4d2c-9a9d-dc1aba94310a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:00.280242 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbfaef-b135-4d2c-9a9d-dc1aba94310a admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:00.290689 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1447/2890] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:37:00.297172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9d80b-0799-4e29-a49c-b367670f3f1d admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:00.314026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9d80b-0799-4e29-a49c-b367670f3f1d admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:00.322123 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec6a37-9564-4609-89e1-b6e1be225164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:00.322773 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec6a37-9564-4609-89e1-b6e1be225164 admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:00.323041 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec6a37-9564-4609-89e1-b6e1be225164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:00.323306 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec6a37-9564-4609-89e1-b6e1be225164 admin admin] PATH_INFO: `/v3/projects/aeffceb9fb554af5b59e986e65b76bdc/users/a6211b7de3ec49c4abc82553233d4d74/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:00.325318 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec6a37-9564-4609-89e1-b6e1be225164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeffceb9fb554af5b59e986e65b76bdc, user_id=a6211b7de3ec49c4abc82553233d4d74, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:00.327934 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec6a37-9564-4609-89e1-b6e1be225164 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:00.338622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1444/2891] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:37:00 2022] PUT /identity/v3/projects/aeffceb9fb554af5b59e986e65b76bdc/users/a6211b7de3ec49c4abc82553233d4d74/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:00.350345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49756aaa-4e2c-40b1-b1ca-a737562d9dd0 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:00.350593 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49756aaa-4e2c-40b1-b1ca-a737562d9dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:00.350859 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49756aaa-4e2c-40b1-b1ca-a737562d9dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:00.358681 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:00.382776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-49756aaa-4e2c-40b1-b1ca-a737562d9dd0 None None] MFA Rules not processed for user `a6211b7de3ec49c4abc82553233d4d74`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:00.384014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-49756aaa-4e2c-40b1-b1ca-a737562d9dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:00.463413 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1448/2892] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:37:00.475074 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec6a37-9564-4609-89e1-b6e1be225164 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:00.477389 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b9ec6a37-9564-4609-89e1-b6e1be225164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:00.561741 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:00.570524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec6a37-9564-4609-89e1-b6e1be225164 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:00.576030 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e10d31-1697-477b-b470-d0d5a4fee827 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:00.576978 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e10d31-1697-477b-b470-d0d5a4fee827 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:00.577293 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e10d31-1697-477b-b470-d0d5a4fee827 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:00.577613 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e10d31-1697-477b-b470-d0d5a4fee827 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:00.590235 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e10d31-1697-477b-b470-d0d5a4fee827 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:00.591695 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e10d31-1697-477b-b470-d0d5a4fee827 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:00.601579 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1445/2893] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:37:00.693159 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49756aaa-4e2c-40b1-b1ca-a737562d9dd0 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:00.704216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49756aaa-4e2c-40b1-b1ca-a737562d9dd0 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:00.708179 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c940b2-f210-475e-9aac-cb76216c19c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:00.708831 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c940b2-f210-475e-9aac-cb76216c19c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:00.709085 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c940b2-f210-475e-9aac-cb76216c19c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:00.709352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c940b2-f210-475e-9aac-cb76216c19c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:00.774627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c940b2-f210-475e-9aac-cb76216c19c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:00.776077 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c940b2-f210-475e-9aac-cb76216c19c1 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:00.785284 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1449/2894] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:37:10.597098 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=6 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:10.780903 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=61 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:10.874645 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e10d31-1697-477b-b470-d0d5a4fee827 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:10.938674 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e10d31-1697-477b-b470-d0d5a4fee827 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:10.944675 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:10.945326 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:10.945596 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:10.945833 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] PATH_INFO: `/v3/users/0aa6ff5413cb49b780ce6dddd6d487eb` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:10.947158 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa6ff5413cb49b780ce6dddd6d487eb)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:10.949040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:10.999269 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa6ff5413cb49b780ce6dddd6d487eb'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.007254 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa6ff5413cb49b780ce6dddd6d487eb'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.021072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa6ff5413cb49b780ce6dddd6d487eb'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.029858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa6ff5413cb49b780ce6dddd6d487eb'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.030517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa6ff5413cb49b780ce6dddd6d487eb'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.036575 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1446/2895] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:37:10 2022] DELETE /identity/v3/users/0aa6ff5413cb49b780ce6dddd6d487eb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:11.041251 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b85c2-ca5d-484d-8068-52e2c97ed4aa None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:11.041528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b85c2-ca5d-484d-8068-52e2c97ed4aa None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:11.041793 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b85c2-ca5d-484d-8068-52e2c97ed4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:11.062005 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-e98b85c2-ca5d-484d-8068-52e2c97ed4aa None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:11.063213 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e98b85c2-ca5d-484d-8068-52e2c97ed4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:11.130226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1450/2896] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:37:11.140493 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:11.141983 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:11.201411 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f834b-c783-4b43-aeb4-dcab63c6c3fb admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:11.205144 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46250967-a8a6-4bc3-973b-ada477ba53e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:11.205747 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46250967-a8a6-4bc3-973b-ada477ba53e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:11.206014 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46250967-a8a6-4bc3-973b-ada477ba53e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:11.206316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46250967-a8a6-4bc3-973b-ada477ba53e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:11.215864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46250967-a8a6-4bc3-973b-ada477ba53e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:11.217309 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46250967-a8a6-4bc3-973b-ada477ba53e3 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:11.226796 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1447/2897] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 15:37:11.503345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b85c2-ca5d-484d-8068-52e2c97ed4aa None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:11.504855 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-e98b85c2-ca5d-484d-8068-52e2c97ed4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:11.518378 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b85c2-ca5d-484d-8068-52e2c97ed4aa None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:11.523248 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:11.523892 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:11.524322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:11.524615 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] PATH_INFO: `/v3/projects/0b094446c5cd402f899c76a8b83a0034` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:11.526065 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b094446c5cd402f899c76a8b83a0034)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:11.527968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:11.581149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b094446c5cd402f899c76a8b83a0034', 'request_id': 'req-c72104d6-ee1e-4409-aa86-dc03ed7cb279'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.590068 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b094446c5cd402f899c76a8b83a0034', 'request_id': 'req-c72104d6-ee1e-4409-aa86-dc03ed7cb279'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.596618 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b094446c5cd402f899c76a8b83a0034', 'request_id': 'req-c72104d6-ee1e-4409-aa86-dc03ed7cb279'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.607558 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] The token cache is being invalidate because project 0b094446c5cd402f899c76a8b83a0034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:37:11.607842 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.608517 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.610096 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1451/2898] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:37:11 2022] DELETE /identity/v3/projects/0b094446c5cd402f899c76a8b83a0034 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:37:11.614282 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46250967-a8a6-4bc3-973b-ada477ba53e3 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:11.625224 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46250967-a8a6-4bc3-973b-ada477ba53e3 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:11.644763 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:11.684824 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32cc05-22a6-43d1-aa0a-8816b038eed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:11.685434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32cc05-22a6-43d1-aa0a-8816b038eed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:11.685701 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32cc05-22a6-43d1-aa0a-8816b038eed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:11.685960 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32cc05-22a6-43d1-aa0a-8816b038eed9 admin admin] PATH_INFO: `/v3/users/a6211b7de3ec49c4abc82553233d4d74` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:11.687096 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32cc05-22a6-43d1-aa0a-8816b038eed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6211b7de3ec49c4abc82553233d4d74)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:11.689026 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32cc05-22a6-43d1-aa0a-8816b038eed9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:11.736133 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bb32cc05-22a6-43d1-aa0a-8816b038eed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6211b7de3ec49c4abc82553233d4d74'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.743290 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bb32cc05-22a6-43d1-aa0a-8816b038eed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6211b7de3ec49c4abc82553233d4d74'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.754857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bb32cc05-22a6-43d1-aa0a-8816b038eed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6211b7de3ec49c4abc82553233d4d74'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.763641 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bb32cc05-22a6-43d1-aa0a-8816b038eed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6211b7de3ec49c4abc82553233d4d74'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.764259 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-bb32cc05-22a6-43d1-aa0a-8816b038eed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6211b7de3ec49c4abc82553233d4d74'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:11.770787 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1448/2899] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:37:11 2022] DELETE /identity/v3/users/a6211b7de3ec49c4abc82553233d4d74 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:12.063936 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:12.065457 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:12.079712 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72104d6-ee1e-4409-aa86-dc03ed7cb279 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:12.138539 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c737f3-8052-4899-84d0-c580357d9ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:12.139035 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c737f3-8052-4899-84d0-c580357d9ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:12.139303 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c737f3-8052-4899-84d0-c580357d9ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:12.139572 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c737f3-8052-4899-84d0-c580357d9ecd admin admin] PATH_INFO: `/v3/projects/aeffceb9fb554af5b59e986e65b76bdc` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:12.140713 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c737f3-8052-4899-84d0-c580357d9ecd admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeffceb9fb554af5b59e986e65b76bdc)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:12.142596 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c737f3-8052-4899-84d0-c580357d9ecd admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:12.174583 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:12.193263 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e3c737f3-8052-4899-84d0-c580357d9ecd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeffceb9fb554af5b59e986e65b76bdc', 'request_id': 'req-e3c737f3-8052-4899-84d0-c580357d9ecd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:12.201915 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e3c737f3-8052-4899-84d0-c580357d9ecd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeffceb9fb554af5b59e986e65b76bdc', 'request_id': 'req-e3c737f3-8052-4899-84d0-c580357d9ecd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:12.208191 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e3c737f3-8052-4899-84d0-c580357d9ecd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeffceb9fb554af5b59e986e65b76bdc', 'request_id': 'req-e3c737f3-8052-4899-84d0-c580357d9ecd'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:12.219942 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e3c737f3-8052-4899-84d0-c580357d9ecd admin admin] The token cache is being invalidate because project aeffceb9fb554af5b59e986e65b76bdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:37:12.220257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e3c737f3-8052-4899-84d0-c580357d9ecd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:12.221012 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-e3c737f3-8052-4899-84d0-c580357d9ecd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:12.222533 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1452/2900] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:37:12 2022] DELETE /identity/v3/projects/aeffceb9fb554af5b59e986e65b76bdc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:12.227517 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c817-f06f-4a21-872c-e2f3390e7070 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:12.227790 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c817-f06f-4a21-872c-e2f3390e7070 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:12.228059 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929c817-f06f-4a21-872c-e2f3390e7070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:12.248277 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-0929c817-f06f-4a21-872c-e2f3390e7070 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:12.249501 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0929c817-f06f-4a21-872c-e2f3390e7070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:12.312492 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1449/2901] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:37:12.320622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f66a4-e84c-46e0-9c39-681cc76e59ea None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:12.320917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f66a4-e84c-46e0-9c39-681cc76e59ea None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:12.321180 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f66a4-e84c-46e0-9c39-681cc76e59ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:12.342055 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-243f66a4-e84c-46e0-9c39-681cc76e59ea None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:12.343300 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-243f66a4-e84c-46e0-9c39-681cc76e59ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:12.351600 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1453/2902] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:37:12.356399 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929c817-f06f-4a21-872c-e2f3390e7070 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:12.367204 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929c817-f06f-4a21-872c-e2f3390e7070 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:12.372010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd604b-23f1-4a88-a190-f4c42f413650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:12.372622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd604b-23f1-4a88-a190-f4c42f413650 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:12.372895 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd604b-23f1-4a88-a190-f4c42f413650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:12.373158 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd604b-23f1-4a88-a190-f4c42f413650 admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:12.373800 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd604b-23f1-4a88-a190-f4c42f413650 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:37:12.374092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd604b-23f1-4a88-a190-f4c42f413650 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:12.375673 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd604b-23f1-4a88-a190-f4c42f413650 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:12.387856 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1450/2903] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:37:12.393276 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f66a4-e84c-46e0-9c39-681cc76e59ea None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:12.404554 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f66a4-e84c-46e0-9c39-681cc76e59ea None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:12.411813 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44b113-f84a-4de7-8591-b7f29da4c03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:12.412686 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b113-f84a-4de7-8591-b7f29da4c03f admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:12.413123 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b113-f84a-4de7-8591-b7f29da4c03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:12.413491 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b113-f84a-4de7-8591-b7f29da4c03f admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:12.414483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44b113-f84a-4de7-8591-b7f29da4c03f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:12.417680 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44b113-f84a-4de7-8591-b7f29da4c03f admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:12.456106 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1454/2904] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:37:12 2022] POST /identity/v3/projects => generated 363 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:37:12.460619 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd604b-23f1-4a88-a190-f4c42f413650 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:12.475662 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd604b-23f1-4a88-a190-f4c42f413650 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:12.513594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:12.550446 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31456360-7d5b-407e-a21a-676ffc6ff157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:12.551100 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31456360-7d5b-407e-a21a-676ffc6ff157 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:12.551374 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31456360-7d5b-407e-a21a-676ffc6ff157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:12.551637 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31456360-7d5b-407e-a21a-676ffc6ff157 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:12.552296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31456360-7d5b-407e-a21a-676ffc6ff157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:12.554284 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31456360-7d5b-407e-a21a-676ffc6ff157 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:12.578439 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1451/2905] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:37:12 2022] POST /identity/v3/users => generated 282 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 15:37:12.582694 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44b113-f84a-4de7-8591-b7f29da4c03f admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:12.593586 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44b113-f84a-4de7-8591-b7f29da4c03f admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:12.598798 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d6c09-87a0-44bc-8501-72f8550a84c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:12.599398 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d6c09-87a0-44bc-8501-72f8550a84c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:12.599697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d6c09-87a0-44bc-8501-72f8550a84c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:12.599959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d6c09-87a0-44bc-8501-72f8550a84c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:12.600581 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d6c09-87a0-44bc-8501-72f8550a84c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:12.602320 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d6c09-87a0-44bc-8501-72f8550a84c6 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:12.612649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1455/2906] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:37:12.617395 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31456360-7d5b-407e-a21a-676ffc6ff157 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:12.628435 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31456360-7d5b-407e-a21a-676ffc6ff157 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:12.633858 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5a4b2-bd6e-45cb-8e00-13c29a73ecc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:12.634440 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5a4b2-bd6e-45cb-8e00-13c29a73ecc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:12.635443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5a4b2-bd6e-45cb-8e00-13c29a73ecc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:12.635720 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5a4b2-bd6e-45cb-8e00-13c29a73ecc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:12.636380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5a4b2-bd6e-45cb-8e00-13c29a73ecc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:12.638024 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5a4b2-bd6e-45cb-8e00-13c29a73ecc7 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:12.649226 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1452/2907] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:37:12.653566 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d6c09-87a0-44bc-8501-72f8550a84c6 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:12.663776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d6c09-87a0-44bc-8501-72f8550a84c6 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:12.668691 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fa635-d9a6-40ad-b293-e2dd3dacbf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:12.669335 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fa635-d9a6-40ad-b293-e2dd3dacbf62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:12.669603 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fa635-d9a6-40ad-b293-e2dd3dacbf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:12.669864 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fa635-d9a6-40ad-b293-e2dd3dacbf62 admin admin] PATH_INFO: `/v3/projects/08a7171cb36644ba980ab121261d41f6/users/69569efdbf774220a8addd2124e53bdd/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:12.685968 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fa635-d9a6-40ad-b293-e2dd3dacbf62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a7171cb36644ba980ab121261d41f6, user_id=69569efdbf774220a8addd2124e53bdd, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:12.688695 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419fa635-d9a6-40ad-b293-e2dd3dacbf62 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:12.703824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1456/2908] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:37:12 2022] PUT /identity/v3/projects/08a7171cb36644ba980ab121261d41f6/users/69569efdbf774220a8addd2124e53bdd/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:12.770999 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510eb180-66b7-4591-9e4e-f70bfaa6666b None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:12.771249 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510eb180-66b7-4591-9e4e-f70bfaa6666b None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:12.771536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510eb180-66b7-4591-9e4e-f70bfaa6666b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:12.806286 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-510eb180-66b7-4591-9e4e-f70bfaa6666b None None] MFA Rules not processed for user `69569efdbf774220a8addd2124e53bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:12.807594 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-510eb180-66b7-4591-9e4e-f70bfaa6666b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:12.877897 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1453/2909] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:37:12.882662 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fa635-d9a6-40ad-b293-e2dd3dacbf62 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:12.946592 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419fa635-d9a6-40ad-b293-e2dd3dacbf62 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:12.951760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b948ea4-23c7-46f6-8dc9-f9cc777db510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:12.952360 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b948ea4-23c7-46f6-8dc9-f9cc777db510 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:12.952621 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b948ea4-23c7-46f6-8dc9-f9cc777db510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:12.952917 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b948ea4-23c7-46f6-8dc9-f9cc777db510 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:12.953538 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b948ea4-23c7-46f6-8dc9-f9cc777db510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:12.955427 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b948ea4-23c7-46f6-8dc9-f9cc777db510 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:12.985981 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1457/2910] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:37:12 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:37:12.990622 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510eb180-66b7-4591-9e4e-f70bfaa6666b None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:13.003228 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510eb180-66b7-4591-9e4e-f70bfaa6666b None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:13.065018 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ddbdf-c28a-4a7b-ba4e-4178ce694846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:13.065666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ddbdf-c28a-4a7b-ba4e-4178ce694846 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:13.065943 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ddbdf-c28a-4a7b-ba4e-4178ce694846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:13.066211 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ddbdf-c28a-4a7b-ba4e-4178ce694846 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:13.066867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ddbdf-c28a-4a7b-ba4e-4178ce694846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:13.068907 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ddbdf-c28a-4a7b-ba4e-4178ce694846 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:13.094438 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1454/2911] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:37:12 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:37:13.098884 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b948ea4-23c7-46f6-8dc9-f9cc777db510 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:13.111316 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b948ea4-23c7-46f6-8dc9-f9cc777db510 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:13.117744 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75226404-b473-4fbd-86cc-9134beeae0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:13.118355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75226404-b473-4fbd-86cc-9134beeae0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:13.118627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75226404-b473-4fbd-86cc-9134beeae0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:13.118897 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75226404-b473-4fbd-86cc-9134beeae0c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:13.120124 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75226404-b473-4fbd-86cc-9134beeae0c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:13.121755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75226404-b473-4fbd-86cc-9134beeae0c8 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:13.133433 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1458/2912] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:37:13.138333 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ddbdf-c28a-4a7b-ba4e-4178ce694846 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:13.148993 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ddbdf-c28a-4a7b-ba4e-4178ce694846 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:13.154941 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c00609-fd6a-4b80-8565-e510bd19ee2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:13.155536 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c00609-fd6a-4b80-8565-e510bd19ee2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:13.155810 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c00609-fd6a-4b80-8565-e510bd19ee2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:13.156081 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c00609-fd6a-4b80-8565-e510bd19ee2b admin admin] PATH_INFO: `/v3/projects/dc48d793ae1f4a0eb41a0f3c0ecdf0bb/users/083085520b9d414d9cad5dcb44618e8a/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:13.172221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c00609-fd6a-4b80-8565-e510bd19ee2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc48d793ae1f4a0eb41a0f3c0ecdf0bb, user_id=083085520b9d414d9cad5dcb44618e8a, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:13.174889 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c00609-fd6a-4b80-8565-e510bd19ee2b admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:13.189832 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1455/2913] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:37:13 2022] PUT /identity/v3/projects/dc48d793ae1f4a0eb41a0f3c0ecdf0bb/users/083085520b9d414d9cad5dcb44618e8a/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:13.195027 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75226404-b473-4fbd-86cc-9134beeae0c8 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:13.205557 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75226404-b473-4fbd-86cc-9134beeae0c8 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:13.226547 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:13.267265 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba80af-68b6-4e70-a38f-0f800bfd3921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:13.267872 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba80af-68b6-4e70-a38f-0f800bfd3921 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:13.268150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba80af-68b6-4e70-a38f-0f800bfd3921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:13.268416 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba80af-68b6-4e70-a38f-0f800bfd3921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:13.269089 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba80af-68b6-4e70-a38f-0f800bfd3921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:13.270694 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba80af-68b6-4e70-a38f-0f800bfd3921 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:13.280729 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1459/2914] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:37:13.285170 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c00609-fd6a-4b80-8565-e510bd19ee2b admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:13.290842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:13.297651 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c00609-fd6a-4b80-8565-e510bd19ee2b admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:13.303192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6776439-24c6-4d15-b78a-2b849ac4a3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:13.304031 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6776439-24c6-4d15-b78a-2b849ac4a3fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:13.304314 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6776439-24c6-4d15-b78a-2b849ac4a3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:13.304598 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6776439-24c6-4d15-b78a-2b849ac4a3fa admin admin] PATH_INFO: `/v3/projects/dc48d793ae1f4a0eb41a0f3c0ecdf0bb/users/083085520b9d414d9cad5dcb44618e8a/roles/ae444a489623434999d758b71ae2dc87` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:13.306885 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6776439-24c6-4d15-b78a-2b849ac4a3fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc48d793ae1f4a0eb41a0f3c0ecdf0bb, user_id=083085520b9d414d9cad5dcb44618e8a, role_id=ae444a489623434999d758b71ae2dc87)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:13.309484 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6776439-24c6-4d15-b78a-2b849ac4a3fa admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:13.320742 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1456/2915] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:37:13 2022] PUT /identity/v3/projects/dc48d793ae1f4a0eb41a0f3c0ecdf0bb/users/083085520b9d414d9cad5dcb44618e8a/roles/ae444a489623434999d758b71ae2dc87 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:13.331312 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd952d3-d45d-474f-b7bb-b71e9f591536 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:13.331591 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd952d3-d45d-474f-b7bb-b71e9f591536 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:13.331861 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd952d3-d45d-474f-b7bb-b71e9f591536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:13.360804 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-0dd952d3-d45d-474f-b7bb-b71e9f591536 None None] MFA Rules not processed for user `083085520b9d414d9cad5dcb44618e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:13.362034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0dd952d3-d45d-474f-b7bb-b71e9f591536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:13.432425 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1460/2916] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:37:13.444092 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6776439-24c6-4d15-b78a-2b849ac4a3fa admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:13.446006 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b6776439-24c6-4d15-b78a-2b849ac4a3fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:13.507167 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6776439-24c6-4d15-b78a-2b849ac4a3fa admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:13.511650 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ccd80-d104-442a-854c-377c09539718 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:13.512246 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ccd80-d104-442a-854c-377c09539718 service cinder] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:13.512514 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ccd80-d104-442a-854c-377c09539718 service cinder] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:13.512804 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ccd80-d104-442a-854c-377c09539718 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:13.522450 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ccd80-d104-442a-854c-377c09539718 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:13.523854 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ccd80-d104-442a-854c-377c09539718 service cinder] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:13.534070 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1457/2917] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:37:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:37:13.618684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd952d3-d45d-474f-b7bb-b71e9f591536 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:13.628951 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd952d3-d45d-474f-b7bb-b71e9f591536 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:13.632446 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63fda8-1e07-4bc0-bdbc-2313c1127c29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:13.633077 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63fda8-1e07-4bc0-bdbc-2313c1127c29 service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:13.633344 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63fda8-1e07-4bc0-bdbc-2313c1127c29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:13.633601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63fda8-1e07-4bc0-bdbc-2313c1127c29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:13.692113 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63fda8-1e07-4bc0-bdbc-2313c1127c29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:13.693478 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63fda8-1e07-4bc0-bdbc-2313c1127c29 service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:13.702766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1461/2918] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:37:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:37:23.529611 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:23.698719 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=80 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:24.171380 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ccd80-d104-442a-854c-377c09539718 service cinder] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:24.236156 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ccd80-d104-442a-854c-377c09539718 service cinder] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:24.241913 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:24.242550 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:24.242842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:24.243117 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] PATH_INFO: `/v3/users/69569efdbf774220a8addd2124e53bdd` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:24.244316 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=69569efdbf774220a8addd2124e53bdd)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:24.246277 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:24.296434 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69569efdbf774220a8addd2124e53bdd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:24.303886 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69569efdbf774220a8addd2124e53bdd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:24.315852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69569efdbf774220a8addd2124e53bdd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:24.326811 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69569efdbf774220a8addd2124e53bdd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:24.327544 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69569efdbf774220a8addd2124e53bdd'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:24.334221 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1458/2919] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:37:24 2022] DELETE /identity/v3/users/69569efdbf774220a8addd2124e53bdd => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:24.339429 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa64ac-046d-4555-a32a-71a0e2c91ae2 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:24.339708 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa64ac-046d-4555-a32a-71a0e2c91ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:24.339979 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa64ac-046d-4555-a32a-71a0e2c91ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:24.361885 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-b1fa64ac-046d-4555-a32a-71a0e2c91ae2 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:24.363083 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b1fa64ac-046d-4555-a32a-71a0e2c91ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:24.425227 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1462/2920] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:37:24.435926 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:24.437566 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:24.506334 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c081d9f2-9d7d-4779-8029-8cda9aa95ebf admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:24.510474 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0586a-462c-4d19-9328-43dbe8362702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:24.511335 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0586a-462c-4d19-9328-43dbe8362702 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:24.511634 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0586a-462c-4d19-9328-43dbe8362702 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:24.511936 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0586a-462c-4d19-9328-43dbe8362702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:24.523981 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0586a-462c-4d19-9328-43dbe8362702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:24.525760 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0586a-462c-4d19-9328-43dbe8362702 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:24.536946 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1459/2921] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:37:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:37:24.817238 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa64ac-046d-4555-a32a-71a0e2c91ae2 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:24.818959 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-b1fa64ac-046d-4555-a32a-71a0e2c91ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:24.832076 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa64ac-046d-4555-a32a-71a0e2c91ae2 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:24.838801 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:24.839483 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:24.839726 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:24.840020 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] PATH_INFO: `/v3/projects/08a7171cb36644ba980ab121261d41f6` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:24.841226 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] RBAC: Authorizing `identity:delete_project(project_id=08a7171cb36644ba980ab121261d41f6)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:24.843127 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:24.892149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08a7171cb36644ba980ab121261d41f6', 'request_id': 'req-0f7da16b-7708-408e-a10a-49f9d98e469d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:24.902530 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08a7171cb36644ba980ab121261d41f6', 'request_id': 'req-0f7da16b-7708-408e-a10a-49f9d98e469d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:24.909993 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08a7171cb36644ba980ab121261d41f6', 'request_id': 'req-0f7da16b-7708-408e-a10a-49f9d98e469d'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:24.923059 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] The token cache is being invalidate because project 08a7171cb36644ba980ab121261d41f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:37:24.923346 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:24.924025 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:24.925568 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1463/2922] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:37:24 2022] DELETE /identity/v3/projects/08a7171cb36644ba980ab121261d41f6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 15:37:24.929918 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0586a-462c-4d19-9328-43dbe8362702 service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:24.940524 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0586a-462c-4d19-9328-43dbe8362702 service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:24.959425 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:25.006064 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80866a7d-391e-4b68-a087-5eabf25ca5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:25.006690 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80866a7d-391e-4b68-a087-5eabf25ca5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:25.006959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80866a7d-391e-4b68-a087-5eabf25ca5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:25.007231 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80866a7d-391e-4b68-a087-5eabf25ca5f9 admin admin] PATH_INFO: `/v3/users/083085520b9d414d9cad5dcb44618e8a` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:25.008464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80866a7d-391e-4b68-a087-5eabf25ca5f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=083085520b9d414d9cad5dcb44618e8a)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:25.010378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80866a7d-391e-4b68-a087-5eabf25ca5f9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:25.059915 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-80866a7d-391e-4b68-a087-5eabf25ca5f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '083085520b9d414d9cad5dcb44618e8a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:25.066821 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-80866a7d-391e-4b68-a087-5eabf25ca5f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '083085520b9d414d9cad5dcb44618e8a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:25.079833 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-80866a7d-391e-4b68-a087-5eabf25ca5f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '083085520b9d414d9cad5dcb44618e8a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:25.089640 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-80866a7d-391e-4b68-a087-5eabf25ca5f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '083085520b9d414d9cad5dcb44618e8a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:25.090402 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-80866a7d-391e-4b68-a087-5eabf25ca5f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '083085520b9d414d9cad5dcb44618e8a'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:25.096842 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1460/2923] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:37:24 2022] DELETE /identity/v3/users/083085520b9d414d9cad5dcb44618e8a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:25.412887 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:25.414627 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:25.428960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7da16b-7708-408e-a10a-49f9d98e469d admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:25.489345 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:25.489977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:25.490253 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:25.490528 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4 admin admin] PATH_INFO: `/v3/projects/dc48d793ae1f4a0eb41a0f3c0ecdf0bb` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:25.492047 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc48d793ae1f4a0eb41a0f3c0ecdf0bb)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:25.493977 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:25.528230 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:25.547454 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc48d793ae1f4a0eb41a0f3c0ecdf0bb', 'request_id': 'req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:25.557976 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc48d793ae1f4a0eb41a0f3c0ecdf0bb', 'request_id': 'req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:25.565776 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc48d793ae1f4a0eb41a0f3c0ecdf0bb', 'request_id': 'req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:25.576816 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4 admin admin] The token cache is being invalidate because project dc48d793ae1f4a0eb41a0f3c0ecdf0bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:37:25.577126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:25.577779 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-b3872ac3-565e-4cb7-ac44-5b87c4e3c4a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:25.579294 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1464/2924] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:37:25 2022] DELETE /identity/v3/projects/dc48d793ae1f4a0eb41a0f3c0ecdf0bb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:25.586666 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653c19b-4b2b-421a-aa2d-40ec10334b31 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:25.587177 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653c19b-4b2b-421a-aa2d-40ec10334b31 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:25.587296 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653c19b-4b2b-421a-aa2d-40ec10334b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:25.612549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-3653c19b-4b2b-421a-aa2d-40ec10334b31 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:25.613814 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-3653c19b-4b2b-421a-aa2d-40ec10334b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:25.684549 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1461/2925] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:37:25.693010 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd8353-170a-4b62-b8ef-bdc6c38c143d None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:25.693291 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd8353-170a-4b62-b8ef-bdc6c38c143d None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:25.693556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd8353-170a-4b62-b8ef-bdc6c38c143d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:25.716216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-c5cd8353-170a-4b62-b8ef-bdc6c38c143d None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:25.717752 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-c5cd8353-170a-4b62-b8ef-bdc6c38c143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:25.727339 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1465/2926] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:37:25.734048 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653c19b-4b2b-421a-aa2d-40ec10334b31 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:25.744754 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653c19b-4b2b-421a-aa2d-40ec10334b31 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:25.752494 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19c30d-044b-4190-8dea-8e12bea5aade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:25.753381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19c30d-044b-4190-8dea-8e12bea5aade admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:25.753809 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19c30d-044b-4190-8dea-8e12bea5aade admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:25.754176 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19c30d-044b-4190-8dea-8e12bea5aade admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:25.755136 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19c30d-044b-4190-8dea-8e12bea5aade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:37:25.755499 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19c30d-044b-4190-8dea-8e12bea5aade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:25.758137 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19c30d-044b-4190-8dea-8e12bea5aade admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:25.773203 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1462/2927] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:37:25.779834 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd8353-170a-4b62-b8ef-bdc6c38c143d None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:25.790684 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd8353-170a-4b62-b8ef-bdc6c38c143d None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:25.798064 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d57676-656d-4e66-83e4-d89421f25573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:25.798938 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d57676-656d-4e66-83e4-d89421f25573 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:25.799355 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d57676-656d-4e66-83e4-d89421f25573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:25.799757 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d57676-656d-4e66-83e4-d89421f25573 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:25.800760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d57676-656d-4e66-83e4-d89421f25573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:25.803876 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d57676-656d-4e66-83e4-d89421f25573 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:25.845322 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1466/2928] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:37:25 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:37:25.849737 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19c30d-044b-4190-8dea-8e12bea5aade admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:25.860401 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19c30d-044b-4190-8dea-8e12bea5aade admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:25.888720 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:25.920200 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d713b4c-0998-4aa2-9bd4-676694037c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:25.920838 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d713b4c-0998-4aa2-9bd4-676694037c06 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:25.921110 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d713b4c-0998-4aa2-9bd4-676694037c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:25.921378 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d713b4c-0998-4aa2-9bd4-676694037c06 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:25.922013 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d713b4c-0998-4aa2-9bd4-676694037c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:25.923923 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d713b4c-0998-4aa2-9bd4-676694037c06 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:25.949038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1463/2929] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:37:25 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:37:25.955906 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d57676-656d-4e66-83e4-d89421f25573 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:25.967440 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d57676-656d-4e66-83e4-d89421f25573 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:25.972521 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8f839-0c2d-47e1-8f32-3a0601831aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:25.973171 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8f839-0c2d-47e1-8f32-3a0601831aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:25.973447 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8f839-0c2d-47e1-8f32-3a0601831aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:25.973715 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8f839-0c2d-47e1-8f32-3a0601831aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:25.974358 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8f839-0c2d-47e1-8f32-3a0601831aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:25.975970 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8f839-0c2d-47e1-8f32-3a0601831aa2 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:25.986462 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1467/2930] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 15:37:25.991143 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d713b4c-0998-4aa2-9bd4-676694037c06 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:26.002505 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d713b4c-0998-4aa2-9bd4-676694037c06 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:26.007957 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3445c-7bc8-4dbd-a9d5-53f3432477f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:26.008567 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3445c-7bc8-4dbd-a9d5-53f3432477f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:26.008864 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3445c-7bc8-4dbd-a9d5-53f3432477f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:26.009128 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3445c-7bc8-4dbd-a9d5-53f3432477f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:26.009819 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3445c-7bc8-4dbd-a9d5-53f3432477f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:26.011393 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3445c-7bc8-4dbd-a9d5-53f3432477f6 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:26.022788 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1464/2931] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:37:26.029697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8f839-0c2d-47e1-8f32-3a0601831aa2 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:26.040392 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8f839-0c2d-47e1-8f32-3a0601831aa2 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:26.046383 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f4fc8-f286-420e-9c8c-e542dc79ec2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:26.047034 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f4fc8-f286-420e-9c8c-e542dc79ec2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:26.047308 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f4fc8-f286-420e-9c8c-e542dc79ec2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:26.047572 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f4fc8-f286-420e-9c8c-e542dc79ec2b admin admin] PATH_INFO: `/v3/projects/c0ff7164e3be4c6cb8535d3f583b8515/users/4cfba733a2e34722a1752477c05ef5e2/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:26.064764 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f4fc8-f286-420e-9c8c-e542dc79ec2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ff7164e3be4c6cb8535d3f583b8515, user_id=4cfba733a2e34722a1752477c05ef5e2, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:26.068688 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f4fc8-f286-420e-9c8c-e542dc79ec2b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:26.093579 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1468/2932] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:37:26 2022] PUT /identity/v3/projects/c0ff7164e3be4c6cb8535d3f583b8515/users/4cfba733a2e34722a1752477c05ef5e2/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:26.105555 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50aefec-cc6d-4c24-9931-cb9773ff1685 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:26.105852 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50aefec-cc6d-4c24-9931-cb9773ff1685 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:26.106105 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50aefec-cc6d-4c24-9931-cb9773ff1685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:26.136959 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-b50aefec-cc6d-4c24-9931-cb9773ff1685 None None] MFA Rules not processed for user `4cfba733a2e34722a1752477c05ef5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:26.138227 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-b50aefec-cc6d-4c24-9931-cb9773ff1685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:26.222739 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1465/2933] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:37:26.237435 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f4fc8-f286-420e-9c8c-e542dc79ec2b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:26.239793 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-6a4f4fc8-f286-420e-9c8c-e542dc79ec2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:26.306473 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f4fc8-f286-420e-9c8c-e542dc79ec2b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:26.310760 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7155422-8beb-43b9-bcb3-34d6db7dd3da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cd51776db3d426ea1bf1133344fa187', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:26.311374 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7155422-8beb-43b9-bcb3-34d6db7dd3da service cinder] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:26.311655 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7155422-8beb-43b9-bcb3-34d6db7dd3da service cinder] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:26.311926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7155422-8beb-43b9-bcb3-34d6db7dd3da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:26.322149 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7155422-8beb-43b9-bcb3-34d6db7dd3da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:26.324126 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7155422-8beb-43b9-bcb3-34d6db7dd3da service cinder] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:26.334393 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1469/2934] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:37:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:37:34.950833 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50aefec-cc6d-4c24-9931-cb9773ff1685 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:35.038548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50aefec-cc6d-4c24-9931-cb9773ff1685 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:35.044381 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:35.045072 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:35.045345 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:35.045608 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] PATH_INFO: `/v3/users/4cfba733a2e34722a1752477c05ef5e2` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:35.046873 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cfba733a2e34722a1752477c05ef5e2)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:35.048867 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:35.097093 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:35.104432 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cfba733a2e34722a1752477c05ef5e2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:35.113443 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cfba733a2e34722a1752477c05ef5e2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:35.126061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cfba733a2e34722a1752477c05ef5e2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:35.136884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cfba733a2e34722a1752477c05ef5e2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:35.137588 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cfba733a2e34722a1752477c05ef5e2'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:35.144082 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1466/2935] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:37:34 2022] DELETE /identity/v3/users/4cfba733a2e34722a1752477c05ef5e2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:35.149822 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1da54-45d0-4ea9-ba47-9ec20ab72449 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:35.150340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1da54-45d0-4ea9-ba47-9ec20ab72449 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:35.150340 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1da54-45d0-4ea9-ba47-9ec20ab72449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:35.177411 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-32f1da54-45d0-4ea9-ba47-9ec20ab72449 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:35.178664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-32f1da54-45d0-4ea9-ba47-9ec20ab72449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:35.232611 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:35.248909 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1470/2936] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 15:37:35.259482 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:35.261192 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:35.321220 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8351b02-061f-4bd5-a835-df85e481fafe admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:35.325199 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbb33b-57c6-4962-b5e7-4274634c2893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:35.325806 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbb33b-57c6-4962-b5e7-4274634c2893 service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:35.326071 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbb33b-57c6-4962-b5e7-4274634c2893 service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:35.326338 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbb33b-57c6-4962-b5e7-4274634c2893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:35.336008 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbb33b-57c6-4962-b5e7-4274634c2893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:35.337458 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbb33b-57c6-4962-b5e7-4274634c2893 service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:35.346948 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1467/2937] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:37:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:37:35.650019 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1da54-45d0-4ea9-ba47-9ec20ab72449 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:35.651606 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-32f1da54-45d0-4ea9-ba47-9ec20ab72449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:35.664698 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1da54-45d0-4ea9-ba47-9ec20ab72449 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:35.669846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8f58b-689e-465c-a405-a96f8b9b37d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:35.670494 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8f58b-689e-465c-a405-a96f8b9b37d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:35.670772 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8f58b-689e-465c-a405-a96f8b9b37d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:35.671040 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8f58b-689e-465c-a405-a96f8b9b37d4 admin admin] PATH_INFO: `/v3/projects/c0ff7164e3be4c6cb8535d3f583b8515` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:35.672257 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8f58b-689e-465c-a405-a96f8b9b37d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ff7164e3be4c6cb8535d3f583b8515)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:35.674262 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8f58b-689e-465c-a405-a96f8b9b37d4 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:35.725056 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4fc8f58b-689e-465c-a405-a96f8b9b37d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ff7164e3be4c6cb8535d3f583b8515', 'request_id': 'req-4fc8f58b-689e-465c-a405-a96f8b9b37d4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:35.736697 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4fc8f58b-689e-465c-a405-a96f8b9b37d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ff7164e3be4c6cb8535d3f583b8515', 'request_id': 'req-4fc8f58b-689e-465c-a405-a96f8b9b37d4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:35.743833 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4fc8f58b-689e-465c-a405-a96f8b9b37d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ff7164e3be4c6cb8535d3f583b8515', 'request_id': 'req-4fc8f58b-689e-465c-a405-a96f8b9b37d4'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:35.754664 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4fc8f58b-689e-465c-a405-a96f8b9b37d4 admin admin] The token cache is being invalidate because project c0ff7164e3be4c6cb8535d3f583b8515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:37:35.754945 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4fc8f58b-689e-465c-a405-a96f8b9b37d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:35.755708 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-4fc8f58b-689e-465c-a405-a96f8b9b37d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:37:35.757305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1471/2938] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:37:35 2022] DELETE /identity/v3/projects/c0ff7164e3be4c6cb8535d3f583b8515 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:35.763605 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb62fab-784a-45ef-b213-7e9d8dae6a02 None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:35.763884 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb62fab-784a-45ef-b213-7e9d8dae6a02 None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:35.764147 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb62fab-784a-45ef-b213-7e9d8dae6a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:35.786578 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-efb62fab-784a-45ef-b213-7e9d8dae6a02 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:35.787782 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-efb62fab-784a-45ef-b213-7e9d8dae6a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:35.857420 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1468/2939] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:37:35.865649 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5d3eb-88f5-4f52-a57d-df3b426fb033 None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:35.866292 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5d3eb-88f5-4f52-a57d-df3b426fb033 None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:35.866556 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5d3eb-88f5-4f52-a57d-df3b426fb033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:35.886564 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-90c5d3eb-88f5-4f52-a57d-df3b426fb033 None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:35.887766 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-90c5d3eb-88f5-4f52-a57d-df3b426fb033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:35.896440 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1472/2940] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:37:35.903277 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb62fab-784a-45ef-b213-7e9d8dae6a02 None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:35.916118 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb62fab-784a-45ef-b213-7e9d8dae6a02 None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:35.921670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e6d51-76fa-42bc-b738-600128f46a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:35.922268 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e6d51-76fa-42bc-b738-600128f46a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:35.922538 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e6d51-76fa-42bc-b738-600128f46a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:35.922806 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e6d51-76fa-42bc-b738-600128f46a3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:35.923458 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e6d51-76fa-42bc-b738-600128f46a3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81865) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 15:37:35.923755 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e6d51-76fa-42bc-b738-600128f46a3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:35.925388 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e6d51-76fa-42bc-b738-600128f46a3c admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:35.937376 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1469/2941] 10.222.0.75 () {58 vars in 1153 bytes} [Fri Sep 23 15:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 15:37:35.942255 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5d3eb-88f5-4f52-a57d-df3b426fb033 None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:35.951738 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5d3eb-88f5-4f52-a57d-df3b426fb033 None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:35.957011 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf756b8-2f21-4813-aca5-28abd9e0c757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:35.957669 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf756b8-2f21-4813-aca5-28abd9e0c757 admin admin] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:35.957960 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf756b8-2f21-4813-aca5-28abd9e0c757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:35.958225 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf756b8-2f21-4813-aca5-28abd9e0c757 admin admin] PATH_INFO: `/v3/projects` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:35.958846 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf756b8-2f21-4813-aca5-28abd9e0c757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:35.960767 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf756b8-2f21-4813-aca5-28abd9e0c757 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:35.993156 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1473/2942] 10.222.0.75 () {60 vars in 1153 bytes} [Fri Sep 23 15:37:35 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:37:35.999466 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e6d51-76fa-42bc-b738-600128f46a3c admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:36.015398 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e6d51-76fa-42bc-b738-600128f46a3c admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:36.075670 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:36.094652 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b922a-69f1-4302-a06f-c1e63c1602f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:36.095604 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b922a-69f1-4302-a06f-c1e63c1602f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:36.096034 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b922a-69f1-4302-a06f-c1e63c1602f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:36.096441 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b922a-69f1-4302-a06f-c1e63c1602f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:36.097512 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b922a-69f1-4302-a06f-c1e63c1602f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:36.099845 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b922a-69f1-4302-a06f-c1e63c1602f8 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:36.132935 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1470/2943] 10.222.0.75 () {60 vars in 1144 bytes} [Fri Sep 23 15:37:35 2022] POST /identity/v3/users => generated 284 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 15:37:36.138578 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf756b8-2f21-4813-aca5-28abd9e0c757 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:36.149802 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf756b8-2f21-4813-aca5-28abd9e0c757 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:36.154384 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02ef6a-2280-4015-b55f-15068416178b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:36.155014 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02ef6a-2280-4015-b55f-15068416178b admin admin] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:36.155362 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02ef6a-2280-4015-b55f-15068416178b admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:36.155622 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e02ef6a-2280-4015-b55f-15068416178b admin admin] PATH_INFO: `/v3/roles` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:36.156259 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e02ef6a-2280-4015-b55f-15068416178b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:36.157886 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e02ef6a-2280-4015-b55f-15068416178b admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:36.168550 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1474/2944] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:37:36.172827 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b922a-69f1-4302-a06f-c1e63c1602f8 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:36.184220 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b922a-69f1-4302-a06f-c1e63c1602f8 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:36.190874 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ca53a-db6b-40d4-bd40-14aa133096e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:36.191471 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ca53a-db6b-40d4-bd40-14aa133096e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:36.191742 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ca53a-db6b-40d4-bd40-14aa133096e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:36.192285 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ca53a-db6b-40d4-bd40-14aa133096e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:36.193202 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ca53a-db6b-40d4-bd40-14aa133096e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:36.194906 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ca53a-db6b-40d4-bd40-14aa133096e3 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:36.206207 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1471/2945] 10.222.0.75 () {58 vars in 1122 bytes} [Fri Sep 23 15:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 15:37:36.210640 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02ef6a-2280-4015-b55f-15068416178b admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:36.220926 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e02ef6a-2280-4015-b55f-15068416178b admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:36.226233 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9c46c-dfb7-40b5-8097-6cc621202225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:36.226875 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9c46c-dfb7-40b5-8097-6cc621202225 admin admin] REQUEST_METHOD: `PUT` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:36.227150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9c46c-dfb7-40b5-8097-6cc621202225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:36.227418 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c9c46c-dfb7-40b5-8097-6cc621202225 admin admin] PATH_INFO: `/v3/projects/ec0b9c61613e47d8be279211aa6e5939/users/cb749475944c4ab3be19d5985e41d494/roles/2921a7146fd9408396ca98a4be4ecd3f` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:36.243072 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9c46c-dfb7-40b5-8097-6cc621202225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec0b9c61613e47d8be279211aa6e5939, user_id=cb749475944c4ab3be19d5985e41d494, role_id=2921a7146fd9408396ca98a4be4ecd3f)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:36.245827 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c9c46c-dfb7-40b5-8097-6cc621202225 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:36.262216 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1475/2946] 10.222.0.75 () {60 vars in 1483 bytes} [Fri Sep 23 15:37:36 2022] PUT /identity/v3/projects/ec0b9c61613e47d8be279211aa6e5939/users/cb749475944c4ab3be19d5985e41d494/roles/2921a7146fd9408396ca98a4be4ecd3f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:37:36.274500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc12918-d089-4067-a0b0-d09f8013220f None None] REQUEST_METHOD: `POST` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:36.274772 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc12918-d089-4067-a0b0-d09f8013220f None None] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:36.275040 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc12918-d089-4067-a0b0-d09f8013220f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:36.316062 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.auth.core [None req-cdc12918-d089-4067-a0b0-d09f8013220f None None] MFA Rules not processed for user `cb749475944c4ab3be19d5985e41d494`. Rule list: `[]` (Enabled: `True`). {{(pid=81865) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:37:36.317548 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-cdc12918-d089-4067-a0b0-d09f8013220f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:36.392958 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1472/2947] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:37:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:37:36.406794 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9c46c-dfb7-40b5-8097-6cc621202225 admin admin] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:37:36.408711 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-72c9c46c-dfb7-40b5-8097-6cc621202225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:37:36.496865 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c9c46c-dfb7-40b5-8097-6cc621202225 admin admin] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:37:36.501150 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64f6bb-500f-49ad-8fa1-5230e95dee61 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac783fbe95ea4b74bd46af415c9734fc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:37:36.501755 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64f6bb-500f-49ad-8fa1-5230e95dee61 service glance] REQUEST_METHOD: `GET` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:37:36.502022 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64f6bb-500f-49ad-8fa1-5230e95dee61 service glance] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:37:36.502288 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64f6bb-500f-49ad-8fa1-5230e95dee61 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:37:36.509601 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:36.513166 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64f6bb-500f-49ad-8fa1-5230e95dee61 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:37:36.515594 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64f6bb-500f-49ad-8fa1-5230e95dee61 service glance] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:37:36.525841 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1476/2948] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:37:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:37:46.389553 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:37:46.529552 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=2 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:38:18.008322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc12918-d089-4067-a0b0-d09f8013220f None None] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:38:18.078410 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc12918-d089-4067-a0b0-d09f8013220f None None] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:38:18.084765 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:38:18.085403 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:38:18.085671 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:38:18.085939 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] PATH_INFO: `/v3/users/cb749475944c4ab3be19d5985e41d494` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:38:18.087115 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb749475944c4ab3be19d5985e41d494)` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:38:18.089061 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:38:18.144020 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb749475944c4ab3be19d5985e41d494'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:38:18.151449 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb749475944c4ab3be19d5985e41d494'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:38:18.163996 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb749475944c4ab3be19d5985e41d494'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:38:18.172692 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb749475944c4ab3be19d5985e41d494'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:38:18.173322 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.notifications [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb749475944c4ab3be19d5985e41d494'} {{(pid=81865) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:38:18.180038 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1473/2949] 10.222.0.75 () {58 vars in 1224 bytes} [Fri Sep 23 15:38:18 2022] DELETE /identity/v3/users/cb749475944c4ab3be19d5985e41d494 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:38:18.184745 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b7114-895b-43c2-bb3f-6ed0192f68ba None None] REQUEST_METHOD: `POST` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:38:18.185293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b7114-895b-43c2-bb3f-6ed0192f68ba None None] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:38:18.185293 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b7114-895b-43c2-bb3f-6ed0192f68ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:38:18.209589 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.auth.core [None req-d31b7114-895b-43c2-bb3f-6ed0192f68ba None None] MFA Rules not processed for user `1d564d82faf2482faf5205415837bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=81864) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 15:38:18.210365 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d31b7114-895b-43c2-bb3f-6ed0192f68ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:38:18.280083 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1477/2950] 10.222.0.75 () {58 vars in 958 bytes} [Fri Sep 23 15:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 15:38:18.291463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:38:18.293492 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:38:18.375193 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8371dd-b28d-40df-85a0-caab160685e9 admin admin] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:38:18.382010 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b215c3-177d-4952-a8e2-3d26929361fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02afdedf8d394bbfa977ab7a22fafea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:38:18.382580 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b215c3-177d-4952-a8e2-3d26929361fd service neutron] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:38:18.382857 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b215c3-177d-4952-a8e2-3d26929361fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:38:18.383194 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b215c3-177d-4952-a8e2-3d26929361fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:38:18.394952 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b215c3-177d-4952-a8e2-3d26929361fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:38:18.397007 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b215c3-177d-4952-a8e2-3d26929361fd service neutron] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:38:18.407519 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1474/2951] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:38:18.714923 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b7114-895b-43c2-bb3f-6ed0192f68ba None None] Authenticating user token {{(pid=81864) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:38:18.716689 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.fernet_utils [None req-d31b7114-895b-43c2-bb3f-6ed0192f68ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81864) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:38:18.730305 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b7114-895b-43c2-bb3f-6ed0192f68ba None None] Validating token access rules against request {{(pid=81864) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:38:18.735525 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8bb57-97f5-43c5-b445-8478da84cd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d564d82faf2482faf5205415837bf68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a6e3ae74e78241ac8bfc0ae2c7103bec', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81864) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:38:18.736352 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8bb57-97f5-43c5-b445-8478da84cd06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:38:18.736620 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8bb57-97f5-43c5-b445-8478da84cd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:38:18.736924 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8bb57-97f5-43c5-b445-8478da84cd06 admin admin] PATH_INFO: `/v3/projects/ec0b9c61613e47d8be279211aa6e5939` {{(pid=81864) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:38:18.738080 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8bb57-97f5-43c5-b445-8478da84cd06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec0b9c61613e47d8be279211aa6e5939)` {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:38:18.740404 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8bb57-97f5-43c5-b445-8478da84cd06 admin admin] RBAC: Authorization granted {{(pid=81864) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:38:18.792009 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-96c8bb57-97f5-43c5-b445-8478da84cd06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec0b9c61613e47d8be279211aa6e5939', 'request_id': 'req-96c8bb57-97f5-43c5-b445-8478da84cd06'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:38:18.801375 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-96c8bb57-97f5-43c5-b445-8478da84cd06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec0b9c61613e47d8be279211aa6e5939', 'request_id': 'req-96c8bb57-97f5-43c5-b445-8478da84cd06'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:38:18.808824 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-96c8bb57-97f5-43c5-b445-8478da84cd06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec0b9c61613e47d8be279211aa6e5939', 'request_id': 'req-96c8bb57-97f5-43c5-b445-8478da84cd06'} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:38:18.821847 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-96c8bb57-97f5-43c5-b445-8478da84cd06 admin admin] The token cache is being invalidate because project ec0b9c61613e47d8be279211aa6e5939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81864) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 15:38:18.822116 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-96c8bb57-97f5-43c5-b445-8478da84cd06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:38:18.822892 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG keystone.notifications [None req-96c8bb57-97f5-43c5-b445-8478da84cd06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81864) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 15:38:18.824514 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: [pid: 81864|app: 0|req: 1478/2952] 10.222.0.75 () {58 vars in 1233 bytes} [Fri Sep 23 15:38:18 2022] DELETE /identity/v3/projects/ec0b9c61613e47d8be279211aa6e5939 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 15:38:28.402665 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:38:28.817522 dv-cinder-855013-3-iscsi devstack@keystone.service[81864]: DEBUG dbcounter [-] [81864] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81864) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 15:41:35.803447 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b215c3-177d-4952-a8e2-3d26929361fd service neutron] Authenticating user token {{(pid=81865) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 15:41:35.805102 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-58b215c3-177d-4952-a8e2-3d26929361fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:41:35.883500 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b215c3-177d-4952-a8e2-3d26929361fd service neutron] Validating token access rules against request {{(pid=81865) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 15:41:35.948272 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dfd53-c684-42a5-88d5-8cf28b63a66d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b8be1ec4a7a45b88d84bd9c48fab407', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27ac51280046479e82faaf3c0b550f72', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81865) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 15:41:35.948881 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dfd53-c684-42a5-88d5-8cf28b63a66d service placement] REQUEST_METHOD: `GET` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 15:41:35.949172 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dfd53-c684-42a5-88d5-8cf28b63a66d service placement] SCRIPT_NAME: `/identity` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 15:41:35.949464 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dfd53-c684-42a5-88d5-8cf28b63a66d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81865) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 15:41:35.951365 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.fernet_utils [None req-0f8dfd53-c684-42a5-88d5-8cf28b63a66d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81865) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 15:41:35.964130 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8dfd53-c684-42a5-88d5-8cf28b63a66d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 15:41:35.965781 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8dfd53-c684-42a5-88d5-8cf28b63a66d service placement] RBAC: Authorization granted {{(pid=81865) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 15:41:35.976463 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: [pid: 81865|app: 0|req: 1475/2953] 10.222.0.75 () {60 vars in 1365 bytes} [Fri Sep 23 15:41:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 15:41:45.971980 dv-cinder-855013-3-iscsi devstack@keystone.service[81865]: DEBUG dbcounter [-] [81865] Writing DB stats keystone:SELECT=58 {{(pid=81865) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}